System

Abstract

A system is provided.SOLUTION: A system, comprising: means for a user to capture or upload a product image; means for the server to analyze the product image and identify a product category; means for the server to obtain a quoted price from a database based on the identified product category; means for the server to detect scratches and stains from the product image and adjust the quoted price; means for the server to suggest the adjusted price and a disposal method to the user; and means for the user to select a next action based on the suggestion.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention relates to the appraisal and disposal of unwanted items. Specifically, it aims to provide a system that accurately appraise the current value of unwanted items owned by users, such as clothes, miscellaneous goods, electronic devices, and furniture, and efficiently proposes disposal methods (selling, keeping, discarding, etc.). This allows users to quickly and accurately decide on the appropriate disposal of unwanted items. In particular, there is a need to improve the accuracy of appraisals by detecting scratches and stains on items and automatically adjusting the price accordingly. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for a user to photograph or upload product images, a means for transmitting the product images to a server, a means for the server to analyze the product images and identify the product category, a means for the server to obtain a market price from a database based on the identified product category, a means for the server to detect scratches or stains from the product images and adjust the market price, a means for the server to suggest the adjusted price and disposal method to the user, and a means for the user to select a next action based on the suggestion. This allows users to easily and accurately appraise unwanted items and select the optimal disposal method. Furthermore, the accuracy of the appraisal can be further improved by using an image processing algorithm to detect scratches and stains and a convolutional neural network to identify the product category.

[0006] "User" refers to a person who uses the system to upload product images and receive appraisals and disposal suggestions.

[0007] "Product image" refers to a photograph or image data of the product for which the user wishes to have it appraised.

[0008] "Server" refers to a computer system that analyzes product images, identifies product categories, obtains market prices, detects scratches and stains, adjusts prices, and makes suggestions to users.

[0009] "Product category" refers to a product classification (e.g., clothing, miscellaneous goods, electronic devices, furniture) identified by product image analysis.

[0010] "Quoted Price" refers to the average price traded in the market, obtained from a database, based on the identified product category and other attributes.

[0011] "Database" refers to a data management system that stores various data, including market prices and product attribute information, and provides this information as needed.

[0012] "Scratch and stain detection" refers to the process of identifying physical defects on the surface of a product through analysis using image processing technology on product images.

[0013] "Price adjustment" refers to adjusting the market price based on the degree of damage or staining detected.

[0014] "Proposal" refers to the act of notifying the user of the appropriate method of disposal (such as selling, keeping, or destroying) including the price analyzed and adjusted by the server.

[0015] "Image processing algorithm" refers to the calculation procedures and programs used to extract features from product images and detect scratches and stains.

[0016] "Convolutional neural network (CNN)" is a type of deep learning, and refers to an artificial intelligence technology that is particularly used to recognize and analyze image data. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system consists of a server, a terminal, and a user. The program processing of this system is explained below in natural language. Specific examples of use are also included.

[0039] System Configuration

[0040] 1. User Device

[0041] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[0042] 2. Server

[0043] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[0044] 3. Database

[0045] This is a system that stores information such as the market price, category, and handling method of a product.

[0046] Program processing flow (overview)

[0047] 1. Initial Setup and User Interface Display

[0048] User: Launches the app and takes or uploads a product image.

[0049] Terminal: Accepts images and sends them to the server.

[0050] 2. Image transmission and preprocessing

[0051] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[0052] Server: Receives images and prepares them for analysis.

[0053] 3. Image Recognition and Product Identification

[0054] Server: Identifies product categories using AI techniques (e.g., convolutional neural networks).

[0055] Server: Retrieve the market price for the identified item from the database.

[0056] 4. Detecting scratches and stains

[0057] Server: Uses image processing algorithms to detect scratches and stains from product images.

[0058] Server: Adjusts the market price based on the degree of damage or dirt detected.

[0059] 5. Generate final price and proposal

[0060] Server: Proposes the adjusted price and the optimal disposal method (sell, keep, or destroy) to the user.

[0061] 6. Displaying the results

[0062] User: Check the proposed results.

[0063] User: Choose next action based on suggestions.

[0064] Specific examples

[0065] Specific examples of product appraisal usage

[0066] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[0067] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[0068] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[0069] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[0070] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[0071] 6. Server: Adjust price to 4000 based on the impact of this stain.

[0072] 7. Server: Generate a suggestion: "This jacket is worth selling. The current market price is about 4000 yen. Would you like to sell it?"

[0073] 8. User: Check the proposal results and select "Sell."

[0074] This system allows users to obtain accurate valuation information for unwanted items and choose the best course of action to dispose of them efficiently.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[0078] Step 2:

[0079] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[0080] Step 3:

[0081] On the device: The resized image data is sent to the server using an HTTP request.

[0082] Step 4:

[0083] Server: Decodes the received image data into an analyzable format.

[0084] Step 5:

[0085] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[0086] Step 6:

[0087] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[0088] Step 7:

[0089] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[0090] Step 8:

[0091] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[0092] Step 9:

[0093] Server: Determines the adjusted price and the proposed disposal method (sell, keep, or destroy), and generates a message to the user. For example, it creates a message saying, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?"

[0094] Step 10:

[0095] User: Check the suggestions displayed on the app screen.

[0096] Step 11:

[0097] User: Selects an action based on the suggestion, for example, "Sell."

[0098] Step 12:

[0099] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] Currently, systems that appraise products and suggest appropriate disposal methods face challenges in accurately identifying product categories and detecting scratches and stains. In particular, the accuracy of image analysis is low, and appraisal results and disposal method suggestions can be inaccurate, making it difficult for users to obtain reliable information. Another problem is the poor usability of users when entering information into the system.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes: a means for a user to photograph or upload a product image; a means for transmitting the product image to the server; a means for the server to analyze the product image and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product image and adjust the market price; a means for the server to suggest the adjusted price and a disposal method to the user; a means for the user to select a next action based on the suggestion; a means for the server to use a generative AI model for image analysis; and a means for the user to provide a specific prompt sentence as input. This enables the user to accurately identify the product category and detect scratches or stains, and to receive a highly reliable appraisal result and a suggestion of an appropriate disposal method.

[0105] "User" refers to an individual or corporation that uses the System to take or upload product images and receive appraisals and disposal method suggestions.

[0106] "Product Image" refers to digital image data that a user photographs or uploads to represent an unwanted item.

[0107] "Server" refers to a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[0108] "Database" refers to an information management system that stores information such as product categories, market prices, and handling methods.

[0109] A "generative AI model" refers to a trained model that uses artificial intelligence technology to perform image analysis and identify product categories.

[0110] A "prompt sentence" refers to a sentence that allows a user to input specific instructions or information into a system.

[0111] A "convolutional neural network" is a type of deep learning algorithm used in image analysis, and refers to a technology that automatically extracts features from images.

[0112] "Image processing algorithm" refers to the computational procedures or programs used to detect flaws or stains from product images.

[0113] "Quoted Price" refers to the expected price in the market for a particular product category, as retrieved from a database.

[0114] A "proposal message" refers to a message generated by the server to present the adjusted price and the optimal disposal method to the user.

[0115] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system is composed of a server, terminals, and users. The program processing of this system is explained below.

[0116] The system hardware will use smartphones and tablets as user devices, a central computer system as a server, and an information management system as a database, while the software will include generative AI models and image processing algorithms.

[0117] First, the user uses a smartphone or tablet device to take a photo of the item they want to appraise or upload an existing image. This operation is performed via a dedicated application installed on the device.

[0118] The user device resizes the received image to an appropriate resolution and sends it to the server, which receives the image and uses a generative AI model (e.g., a convolutional neural network) to identify the product category. It then retrieves the market price of the identified product from a database.

[0119] The server then uses image processing algorithms to detect scratches and stains from the product images. Based on this information, the server adjusts the market price. Finally, the server generates a proposal message suggesting the adjusted price and the optimal disposal method (e.g., sell, keep, or destroy) and sends it to the user's device.

[0120] The user can check the suggestion message displayed on the terminal and select the next action based on the suggestion. An example of a specific prompt sentence is, "I uploaded a photo of an old jacket. Please check for defects and stains and let me know the market price."

[0121] Using this system, users can obtain accurate appraisal information for unwanted items and select the optimal action for efficient disposal. For example, if a user wants to have an old jacket appraised, they launch the app, take a photo, and upload it. The system analyzes the image and identifies the jacket's category and market price. It also detects a small stain on the chest, adjusts the price, and suggests selling the item. In this way, the present invention can provide users with highly accurate appraisals and convenient disposal method suggestions.

[0122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0123] Step 1:

[0124] A user takes or uploads a product image.

[0125] Input: Product images taken by a user using a smartphone or tablet, or existing image data.

[0126] How it works: A user launches the app and either takes a photo of the item they want to appraise or selects and uploads an image from their gallery.

[0127] Output: The product image is displayed on the terminal and the button to proceed to the next step is activated.

[0128] Step 2:

[0129] The device preprocesses the uploaded image and sends it to the server.

[0130] Input: A user-uploaded image.

[0131] Data processing: The device resizes the received image to an appropriate resolution (e.g., 1024x768 pixels).

[0132] How it works: The device resizes the image and compresses it while preserving quality.

[0133] Output: The preprocessed image is sent to the server.

[0134] Step 3:

[0135] The server analyzes the image and identifies the product category.

[0136] Input: The preprocessed image received by the server.

[0137] Data computation: The server performs image analysis to identify product categories using a generative AI model (e.g., convolutional neural network).

[0138] How it works: A convolutional neural network extracts features from an image and matches them with information in a database.

[0139] Output: Identified product category (e.g., "Brand name jacket (2005 model)").

[0140] Step 4:

[0141] The server retrieves market prices from a database based on the identified product category.

[0142] Input: Identified product category.

[0143] Data acquisition: The server searches the database for the market price corresponding to the product category and acquires it.

[0144] How it works: The server generates a database query to retrieve detailed product information, including price quotes.

[0145] Output: Market price of the item (e.g., 5000 yen).

[0146] Step 5:

[0147] The server detects scratches and stains from product images and adjusts the market price.

[0148] Input: Product image and quote price.

[0149] Data processing: The server uses image processing algorithms to detect scratches and stains in the product images and adjust the market price.

[0150] How it works: Image analysis algorithms detect defects in images and recalculate prices based on that information.

[0151] Output: The adjusted price (e.g., $4000).

[0152] Step 6:

[0153] The server proposes the adjusted price and disposal method to the user.

[0154] Input: Adjusted price.

[0155] Data Generation: The server generates a proposal message containing the adjusted price and the optimal disposal method.

[0156] Behavior: The server creates a suggestion message that reads, "This jacket is recommended for sale. The current market price is about $40.00. Would you like to sell it?"

[0157] Output: The proposal message is sent to the terminal.

[0158] Step 7:

[0159] The user selects the next action based on the suggestions.

[0160] Input: The suggestion message displayed on the terminal.

[0161] Action: The user reviews the proposal and selects "Sell."

[0162] Output: The user's selection is communicated to the server, and the next action is performed.

[0163] (Application example 1)

[0164] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0165] Conventional systems that assess unwanted products and suggest disposal methods do not support frequently used ingredients or food delivery services. Furthermore, there are no tools that can quickly assess the freshness and condition of ingredients in the refrigerator and suggest optimal ways to use or dispose of them, leaving users without a way to easily reduce food waste. This results in a large amount of food waste within the home, which increases costs and puts a strain on the environment.

[0166] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0167] In this invention, the server includes: a means for a user to photograph or upload product images; a means for transmitting the product images to the server; a means for the server to analyze the product images and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product images and adjust the market price; a means for a user to photograph or upload an image of an ingredient; a means for transmitting the ingredient image to the server; a means for the server to analyze the ingredient image and identify a food category; a means for the server to obtain an expiration date and an optimal usage method from a database based on the identified food category; and a means for the server to detect the freshness and condition of the ingredient from the ingredient image and adjust the usage method. This allows a user to easily check the freshness of ingredients in the refrigerator and quickly find optimal usage and disposal methods to reduce waste.

[0168] A "user terminal" is a device that a user uses to take or upload images of products or ingredients.

[0169] The "server" is a central computer system that analyzes images of products and ingredients, identifies categories, adjusts prices, and detects freshness and condition.

[0170] A "database" is a system that stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[0171] "Product category" refers to the type or classification of a product identified by the server through analysis.

[0172] "Quoted Price" is the market price of the commodity obtained from the database.

[0173] "Scratches and stains" refer to external defects and stains detected from product images.

[0174] "Food image" is an image taken or uploaded by the user to understand the condition of the food in the refrigerator.

[0175] "Freshness" refers to the state of freshness of ingredients and storage conditions.

[0176] "Use by date" is the expiration date of the ingredient obtained from the database.

[0177] The "optimal use" is a method of use suggested by the server based on the freshness and condition of the ingredients.

[0178] The "adjusted price" is the adjusted market price calculated by the server based on the results of detecting scratches and stains.

[0179] The "suggestion result" is information about the adjusted price and how to use the ingredients that is displayed to the user.

[0180] An "action" is the next operation that the user selects based on the suggested results.

[0181] This invention is a system that analyzes the status of unwanted products and food items stored in a user's refrigerator and suggests optimal disposal and usage methods. The system is composed of a server, a user terminal, and a database.

[0182] System Configuration

[0183] User terminal

[0184] It is a device such as a smartphone or tablet that provides an interface for taking and uploading images of products and ingredients and operating applications.

[0185] server

[0186] It is a central computer system that analyzes images of products and ingredients, identifies categories, obtains market prices, detects scratches, dirt, freshness and condition, adjusts prices and suggests ways to use and dispose of them. Specifically, it makes full use of convolutional neural networks (CNN) using TENSORFLOW (registered trademark) and Keras.

[0187] Database

[0188] This system stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[0189] Program processing flow

[0190] User terminal

[0191] The user takes or uploads an image of the product or ingredient.

[0192] The image is resized to the appropriate resolution and sent to the server.

[0193] server

[0194] The server receives the image and analyzes it using a convolutional neural network (CNN).

[0195] For products, the system identifies the category, obtains the market price, detects scratches and stains, and proposes an adjusted price and disposal method.

[0196] In the case of food ingredients, the system identifies the food category, obtains the expiration date and optimal usage method, detects the freshness and condition, and suggests how to use and dispose of the food.

[0197] Specific examples

[0198] Specific examples of product appraisal usage

[0199] 1. The user takes a photo of an old jacket (e.g., a 2005 model from Brand A), launches the app, and uploads the image.

[0200] 2. The device resizes the image and sends it to the server.

[0201] 3. The server analyzes the image, determines the market price of the jacket, detects scratches and stains, and calculates the adjusted price.

[0202] 4. The server generates a suggestion such as "We recommend selling this jacket. The market price is about 4,000 yen. Would you like to sell it?" and displays the suggestion result to the user.

[0203] 5. The user selects "Sell" based on the proposed results.

[0204] Food ingredient management example

[0205] 1. The user takes a photo of an apple in the refrigerator, launches the app and uploads the image.

[0206] 2. The device resizes the image and sends it to the server.

[0207] 3. The server analyzes the image, identifies the apple category, determines its freshness, and retrieves expiration date information from a database.

[0208] 4. The server generates a suggestion such as "This apple is fresh. We recommend eating it raw." and displays the suggestion result to the user.

[0209] 5. The user selects the next action to take based on the suggested results.

[0210] Prompt Sentence Examples

[0211] "Just take a photo of the food in your refrigerator and upload it to the app. The system will automatically determine the freshness of the food and tell you the best way to use it. For example, if you upload a photo of an apple, it will suggest, 'This apple is fresh. We recommend eating it raw.'"

[0212] As described above, the present invention provides a system that allows users to efficiently manage goods and ingredients by analyzing images of the goods and ingredients and proposing optimal disposal and usage methods.

[0213] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0214] Step 1:

[0215] The user takes or uploads an image of the product or ingredient.

[0216] Input: An image of a product or ingredient.

[0217] Specific operation: The user uses a user device such as a smartphone or tablet to take a photo of the target product or ingredient or upload an existing image.

[0218] Output: Image data captured or uploaded.

[0219] Step 2:

[0220] The terminal resizes the product or ingredient image to an appropriate resolution and transmits it to the server.

[0221] Input: Image data captured or uploaded.

[0222] Specific operation: The device resizes the image resolution to, for example, 1024x768 pixels and sends the image data to the server.

[0223] Output: The resized image data.

[0224] Step 3:

[0225] The server receives the images and analyzes them using a convolutional neural network (CNN).

[0226] Input: The resized image data.

[0227] Specific operation: An AI model (using TensorFlow and Keras) installed on the server reads the image and analyzes it using CNN to identify the product or ingredient category.

[0228] Output: Category information of products and ingredients.

[0229] Step 4:

[0230] The server retrieves the market price and expiration date from the database based on the identified category.

[0231] Input: Category information of product or ingredients.

[0232] Specific operation: The server accesses the database and retrieves the market price based on the identified product category, the expiration date based on the food ingredient category, and the latest information.

[0233] Output: Market price information for products or expiration date information for ingredients.

[0234] Step 5:

[0235] The server detects scratches and dirt from product images and detects freshness and condition from food ingredient images.

[0236] Input: Resized image data and category information.

[0237] How it works: The server uses image processing algorithms to detect scratches and dirt from product images, and freshness and condition from food images.

[0238] Output: Information on detected scratches and stains, freshness and condition.

[0239] Step 6:

[0240] Based on the detection results, the server will suggest product price adjustments and optimal ways to use and dispose of ingredients.

[0241] Input: Market price information or expiration date information, damage or stain information, freshness or condition information.

[0242] What it does: The server adjusts the market price based on the results of any damage or stains detected, and determines the best way to use or dispose of the ingredients.

[0243] Output: Adjusted price or recommendations on optimal usage and disposal methods.

[0244] Step 7:

[0245] The user selects the next action based on the suggestions.

[0246] Input: Proposal information.

[0247] Specific operation: The user checks the proposal results on their device and selects the next action (e.g., sell the product, use the ingredients for a delicious meal, or discard if unnecessary).

[0248] Output: The result of the selection of the next action to be taken.

[0249] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0250] This invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained in natural language below. Specific examples of use are also included.

[0251] System Configuration

[0252] 1. User Device

[0253] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[0254] 2. Server

[0255] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[0256] 3. Database

[0257] This is a system that stores information such as the market price, category, and handling method of a product.

[0258] 4. Emotion Engine

[0259] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[0260] Program processing flow (overview)

[0261] 1. Initial Setup and User Interface Display

[0262] User: Launch the app and take or upload a photo of the item you want appraised.

[0263] Terminal: Receives the image and sends it to the server.

[0264] 2. Image transmission and preprocessing

[0265] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[0266] Server: Receives images and prepares them for analysis.

[0267] 3. Image Recognition and Product Identification

[0268] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[0269] Server: Retrieve the market price for the identified item from the database.

[0270] 4. Detecting scratches and stains

[0271] Server: Uses image processing algorithms to detect scratches and stains from product images.

[0272] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[0273] 5. Emotion Analysis

[0274] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[0275] 6. Proposal generation based on final price and sentiment

[0276] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[0277] 7. Displaying the results

[0278] User: Check the proposed results.

[0279] User: Choose next action based on suggestions.

[0280] Specific examples

[0281] Specific use cases for product appraisal and emotion recognition

[0282] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[0283] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[0284] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[0285] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[0286] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[0287] 6. Server: Adjust price to 4000 based on the impact of this stain.

[0288] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[0289] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[0290] 9. User: Check the proposal results and select "Sell" or "Keep."

[0291] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[0292] The processing flow will be explained below.

[0293] Step 1:

[0294] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[0295] Step 2:

[0296] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[0297] Step 3:

[0298] On the device: The resized image data is sent to the server using an HTTP request.

[0299] Step 4:

[0300] Server: Decodes the received image data into an analyzable format.

[0301] Step 5:

[0302] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[0303] Step 6:

[0304] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[0305] Step 7:

[0306] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[0307] Step 8:

[0308] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[0309] Step 9:

[0310] Emotion Engine: Analyzes facial expressions in real time from the user's camera to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[0311] Step 10:

[0312] Server: Based on the user's emotional information obtained from the emotion engine, the server proposes an adjusted price and the optimal disposal method (sell, keep, or destroy). For example, if the user shows signs of being emotionally reluctant to sell, the server will suggest keeping the item.

[0313] Step 11:

[0314] Server: Generates a suggestion message to the user, such as "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" or "Or, consider keeping it," and displays it on the screen.

[0315] Step 12:

[0316] User: Check the proposed results.

[0317] Step 13:

[0318] User: Selects an action based on the suggestion, for example, "sell" or "keep."

[0319] Step 14:

[0320] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[0321] Example 2

[0322] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] Conventional unwanted item appraisal systems have the problem that it is difficult to accurately evaluate the condition of products and do not provide personalized suggestions that take the user's emotions into consideration, resulting in low user satisfaction. This invention aims to achieve more accurate appraisals and suggestions that result in higher user satisfaction by combining product image analysis and an emotion engine.

[0324] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing a product image and identifying a product category, a means for acquiring a market price from a database based on the identified product category, a means for detecting scratches or stains from the product image and adjusting the market price, a means for analyzing the user's emotional state using an emotion engine, and a means for adjusting the proposal content based on the user's emotional state. This enables more accurate appraisals and personalized proposals that take the user's emotions into consideration.

[0325] "Product image" refers to a photograph or image data of the product for which the user wishes to have it appraised.

[0326] A "server" is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and generates suggestions in cooperation with an emotion engine.

[0327] "Product category" is information that indicates the type or classification of a specific product, and by identifying it, it becomes possible to obtain the market price.

[0328] "Quoted Price" refers to the prevailing price in the market for a particular commodity, and is obtained from a database.

[0329] A "database" refers to an information system that stores information such as the market price of a product, category information, and handling instructions.

[0330] "Scratches and stains" refers to physical damage or stains on the surface of the product, which are factors that affect the value of the product.

[0331] An "emotion engine" refers to an algorithm or system that analyzes a user's facial expressions and voice to recognize their emotional state.

[0332] "Adjusting the proposal" refers to optimizing the proposal for how to dispose of the product and the final price based on the user's emotional state.

[0333] An "action" refers to a decision or action (e.g., sell, keep, or discard) that a user makes based on a suggestion from the server.

[0334] The present invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained below in natural language.

[0335] System Configuration

[0336] 1. User Device

[0337] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[0338] 2. Server

[0339] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[0340] 3. Database

[0341] This is a system that stores information such as the market price, category, and handling method of a product.

[0342] 4. Emotion Engine

[0343] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[0344] Program Processing Overview

[0345] The program operates as follows.

[0346] 1. Initial Setup and User Interface Display

[0347] User: Launch the app and take or upload a photo of the item you want appraised.

[0348] Terminal: Receives the image and sends it to the server.

[0349] 2. Image transmission and preprocessing

[0350] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[0351] Server: Receives images and prepares them for analysis.

[0352] 3. Image Recognition and Product Identification

[0353] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[0354] Server: Retrieve the market price for the identified item from the database.

[0355] 4. Detecting scratches and stains

[0356] Server: Uses image processing algorithms to detect scratches and stains from product images.

[0357] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[0358] 5. Emotion Analysis

[0359] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[0360] 6. Proposal generation based on final price and sentiment

[0361] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[0362] 7. Displaying the results

[0363] User: Check the proposed results.

[0364] User: Choose next action based on suggestions.

[0365] Specific examples

[0366] Specific use cases for product appraisal and emotion recognition

[0367] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[0368] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[0369] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[0370] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[0371] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[0372] 6. Server: Adjust price to 4000 based on the impact of this stain.

[0373] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[0374] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[0375] 9. User: Check the proposal results and select "Sell" or "Keep."

[0376] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[0377] Specific prompt examples

[0378] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[0379] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0380] Step 1:

[0381] Initial Setup and User Interface Display

[0382] User: Launches the app and takes a photo of the item they wish to have appraised. The input is a photo of the item. Specifically, the user taps the smartphone screen to launch the app, selects camera mode, and takes a photo. The output is the captured photo data.

[0383] Step 2:

[0384] Image transmission and preprocessing

[0385] Terminal: Resizes a captured photo to a specified resolution (for example, 1024x768 pixels). The input is the captured photo data, which undergoes resolution conversion. Specifically, the application changes the image size internally. The output is the resized image data.

[0386] Device: Sends the resized image to the server. The input is the resized image data, which goes through a process of being sent to the server. Specifically, the device uploads the image to the server using an HTTP request. The output is the image data sent to the server.

[0387] Step 3:

[0388] Image recognition and product identification

[0389] Server: Prepares the received image for analysis. The input is the transmitted image data, which is converted into an internal format so that it can be analyzed. The output is image data that can be analyzed.

[0390] Server: Identifies product categories from images using a convolutional neural network (CNN). The input is analyzable image data, features are extracted using a CNN model, and product categories are identified using a classification model. Specifically, the CNN is executed to identify categories such as "2005 model jackets from a certain brand." The output is the identified product category information.

[0391] Server: Retrieves the market price from the database based on the identified product category. The input is the product category information, and a database query is made based on this. The output is the market price (e.g., 5000 yen).

[0392] Step 4:

[0393] Detecting scratches and stains

[0394] Server: Uses image processing algorithms to detect scratches and stains from product images. The input is analyzable image data, and the image processing algorithm identifies areas of physical damage or stains. Specifically, the algorithm detects stains on the chest of a jacket. The output is information about the detected scratches and stains (area and degree).

[0395] Server: Adjusts the market price based on the detected scratches and stains. The input is the scratch and stain information and the market price, and a comprehensive evaluation is performed to calculate the final price. Specifically, the market price is adjusted to 4000 yen due to the influence of stains. The output is the final price after adjustment.

[0396] Step 5:

[0397] Emotion Analysis

[0398] Emotion engine: Analyzes facial expressions from the user's camera footage in real time to identify the user's emotional state. The input is the user's camera footage, to which an expression analysis algorithm is applied. Specifically, it analyzes the user's facial expressions when viewing the assessment results and determines whether the user is expressing happiness, sadness, surprise, etc. The output is information about the user's emotional state.

[0399] Emotion engine: Analyzes the user's voice to enhance emotional information. The input is the user's voice data, and the voice analysis algorithm supplements the emotional state. The output is integrated emotional state information.

[0400] Step 6:

[0401] Generate offers based on final price and sentiment

[0402] Server: Generates individual proposals based on the adjusted price and the user's emotional information. The input is the final price and the integrated emotional state information. Specifically, the server generates a proposal based on the adjusted price (4000 yen) and the emotional information, such as "We recommend selling this jacket, and the current market price is about 4000 yen. Would you like to sell it?" If the emotion is negative, it also presents an alternative suggestion such as "One option is to consider keeping it." The output is the generated proposal.

[0403] Step 7:

[0404] Displaying the results

[0405] User: Checks the proposal results from the server through the application. The input is the proposal content from the server. The specific operation is that the proposal results are displayed on the smartphone screen. The output is the confirmed proposal results.

[0406] User: Selects the next action based on the suggestion. The input is the suggestion result, and based on that, the user selects an action such as "sell" or "keep." Specific actions are performed by the user tapping an option on the screen. The output is the selected action.

[0407] (Example prompt)

[0408] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[0409] (Application example 2)

[0410] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0411] Conventional food delivery services have been unable to provide personalized menu suggestions or special offers based on the user's emotional state, which makes it difficult for users to receive appropriate service that matches their mood and emotions at the time, potentially resulting in lower satisfaction.

[0412] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for identifying the user's emotion and adjusting the proposal content based on the identification result, means for analyzing product images and identifying the product category, and means for retrieving market prices from a database based on the identified product category. This makes it possible to provide personalized menu suggestions and special offers based on the user's emotional state.

[0413] "Means for users to take or upload product images" refers to devices or software that provide a function for users to take product images and upload them to a server through an application.

[0414] The "means for transmitting the product image to the server" is a communication interface for transmitting the photographed or uploaded product image to the server via the Internet.

[0415] "Means for the server to analyze the product image and identify the product category" refers to a process in which the server uses an image analysis algorithm on the product image received to identify the category to which the product belongs.

[0416] The "means for the server to obtain market price information from the database based on the identified product category" is a process in which the server obtains market price information that matches the identified product category from the database.

[0417] "Means for the server to detect scratches or stains from product images and adjust the market price" refers to a process in which the server analyzes product images, detects the presence or absence of scratches or stains and their extent, and adjusts the market price.

[0418] The "means for the server to propose the adjusted price and disposal method to the user" is a process in which the server proposes to the user, along with the adjusted price information, a disposal method such as whether to sell, keep, or discard the product.

[0419] The "means for the user to select the next action based on the proposal" is an interface that allows the user to receive a proposal from the server and select the next action to be taken (sell, keep, discard, etc.).

[0420] "Means for identifying the user's emotions and adjusting the content of suggestions based on the identification results" refers to a process in which an emotion engine is used to identify emotions from the user's facial expressions and voice, and the content of suggestions is personalized based on the results.

[0421] The present invention relates to a personalized menu recommendation system that utilizes an emotion engine to improve user experience in food delivery services. Specific embodiments for implementing this system will be described below.

[0422] 1. System Configuration

[0423] User terminal

[0424] A device used by a user, such as a smartphone or tablet, that includes a camera and microphone, allowing the user to register emotions through facial expressions and voice, and has an application installed on it that provides an interface for launching the application and placing a delivery order.

[0425] server

[0426] This is a central computer system for analyzing product images, retrieving market prices from a database, detecting scratches and stains, adjusting prices, suggesting disposal methods, and making suggestions based on user emotion recognition. This includes an emotion engine, image analysis algorithms, and database access.

[0427] Database

[0428] This is a system that stores information such as the market price, category, and handling method of a product.

[0429] Emotion Engine

[0430] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[0431] 2. Hardware and software used

[0432] Hardware

[0433] Smartphones and tablets (with cameras and microphones)

[0434] software

[0435] Emotion engine: OpenCV (facial expression analysis), Google (registered trademark) Speech-to-Text API (voice analysis)

[0436] Database: MongoDB (Management of menu / emotion pairing data)

[0437] Server: AWS (registered trademark) Lambda (serverless environment), Amazon RDS (data storage)

[0438] 3. Data processing and calculation

[0439] User terminal

[0440] Emotional data is collected by users showing facial expressions to the camera and inputting voice data through a microphone, and this data is sent to a server via the application.

[0441] Emotion analysis

[0442] The facial expression data received by the server is analyzed by OpenCV, and the voice data is converted to text using the Google Speech-to-Text API, which allows for specific identification of the user's emotional state.

[0443] Database Access and Proposal Generation

[0444] The emotion engine retrieves appropriate menus from the database according to the user's emotional state, for example, suggesting menus that match a specific emotional state (e.g., if you are tired, suggesting menus that will cheer you up).

[0445] Suggestions for users

[0446] The server sends the menu suggestions along with the adjusted price information to the user's device, which is then displayed on the application to help the user choose the best course of action.

[0447] 4. Specific Examples

[0448] Specific examples of menu suggestions

[0449] 1. The user smiles at the camera.

[0450] 2. Saying "I'm tired today" on the microphone.

[0451] 3. OpenCV analyzes whether or not a smile is present and scores the degree of joy.

[0452] 4. The Google Speech-to-Text API converts "I'm tired today" into text and feeds it back into the emotion engine.

[0453] 5. From the database, a "nutritious chicken salad bowl" is suggested as a menu suitable for the "tired" and "happy" scores.

[0454] 6. The suggested menu is displayed within the app and the user taps the "Order" button.

[0455] Examples of prompt statements

[0456] This prompt is used for the generative AI model:

[0457] Suggest cheer-up foods when the user says "I'm feeling down today" and smiles at the camera. Describe how your system personalizes the suggestions and reassessss whether the user is satisfied with them. Use an emotion engine to analyze the user's emotional feedback in real time and adjust the suggestions accordingly. Use smartphone hardware and simulate the processing flow using OpenCV and the Google Speech-To-Text API.

[0458] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0459] Step 1:

[0460] The user launches the food delivery app. The user shows their facial expression in front of the camera and inputs voice using the microphone. The input data is the user's facial image and voice input. The facial image and voice input are saved on the device and prepared for transmission to the server.

[0461] Step 2:

[0462] The device sends the facial expression images and voice input data it has saved to the server. The data input here are facial expression images and voice files, and communication is performed to send them to the server. This allows the server to begin analyzing the user's emotional data.

[0463] Step 3:

[0464] The server analyzes the received facial image using OpenCV and identifies the emotion from the user's facial expression. The input is the facial image, and the output is an emotion score (e.g., joy, sadness, surprise, etc.). The OpenCV algorithm extracts facial features and scores the emotion based on them.

[0465] Step 4:

[0466] The server converts the received voice data into a string using the Google Speech-to-Text API. The input is voice data and the output is text. Speech recognition technology converts the voice data into text data and provides additional information about the user's emotions.

[0467] Step 5:

[0468] The server combines the results of facial expression analysis and voice analysis to determine the overall emotional state. The input is the emotion score and text data, and the output is the overall emotional state (e.g., tired but happy). The emotion engine combines these data to determine the emotional state with greater accuracy.

[0469] Step 6:

[0470] The server selects an appropriate menu from a database based on the overall emotional state. The input is the emotional state, and the output is the corresponding menu information (e.g., candidates for energizing foods). A database search is performed to obtain menus that match the emotion.

[0471] Step 7:

[0472] The server sends the selected menu information to the terminal. The input here is the menu information, and the output is a menu suggestion that is displayed on the user's terminal. The suggested menu and the reason for it are displayed in the user application.

[0473] Step 8:

[0474] The user reviews and selects a menu suggestion from the server. The input here is the suggested menu, and the output is the user's choice (e.g., confirming the order). The user browses the suggested menu on the app and taps a button to confirm the order.

[0475] Step 9:

[0476] The server stores the user's selections and processes the order. The input is the user's selections and the output is the actual order data. The server stores the order information and issues the order to the food delivery service.

[0477] Step 10:

[0478] After the server places an order, it collects user feedback and adds it to the training data to improve the accuracy of the emotion engine. The input here is user feedback, and the output is an updated emotion data model. User feedback is collected periodically and the emotion engine model is updated.

[0479] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0480] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0481] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0482] [Second embodiment]

[0483] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0484] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0485] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0486] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0487] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0488] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0489] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0490] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0491] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0492] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0493] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0494] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0495] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system consists of a server, a terminal, and a user. The program processing of this system is explained below in natural language. Specific examples of use are also included.

[0496] System Configuration

[0497] 1. User Device

[0498] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[0499] 2. Server

[0500] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[0501] 3. Database

[0502] This is a system that stores information such as the market price, category, and handling method of a product.

[0503] Program processing flow (overview)

[0504] 1. Initial Setup and User Interface Display

[0505] User: Launches the app and takes or uploads a product image.

[0506] Terminal: Accepts images and sends them to the server.

[0507] 2. Image transmission and preprocessing

[0508] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[0509] Server: Receives images and prepares them for analysis.

[0510] 3. Image Recognition and Product Identification

[0511] Server: Identifies product categories using AI techniques (e.g., convolutional neural networks).

[0512] Server: Retrieve the market price for the identified item from the database.

[0513] 4. Detecting scratches and stains

[0514] Server: Uses image processing algorithms to detect scratches and stains from product images.

[0515] Server: Adjusts the market price based on the degree of damage or dirt detected.

[0516] 5. Generate final price and proposal

[0517] Server: Proposes the adjusted price and the optimal disposal method (sell, keep, or destroy) to the user.

[0518] 6. Displaying the results

[0519] User: Check the proposed results.

[0520] User: Choose next action based on suggestions.

[0521] Specific examples

[0522] Specific examples of product appraisal usage

[0523] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[0524] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[0525] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[0526] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[0527] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[0528] 6. Server: Adjust price to 4000 based on the impact of this stain.

[0529] 7. Server: Generate a suggestion: "This jacket is worth selling. The current market price is about 4000 yen. Would you like to sell it?"

[0530] 8. User: Check the proposal results and select "Sell."

[0531] This system allows users to obtain accurate valuation information for unwanted items and choose the best course of action to dispose of them efficiently.

[0532] The processing flow will be explained below.

[0533] Step 1:

[0534] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[0535] Step 2:

[0536] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[0537] Step 3:

[0538] On the device: The resized image data is sent to the server using an HTTP request.

[0539] Step 4:

[0540] Server: Decodes the received image data into an analyzable format.

[0541] Step 5:

[0542] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[0543] Step 6:

[0544] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[0545] Step 7:

[0546] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[0547] Step 8:

[0548] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[0549] Step 9:

[0550] Server: Determines the adjusted price and the proposed disposal method (sell, keep, or destroy), and generates a message to the user. For example, it creates a message saying, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?"

[0551] Step 10:

[0552] User: Check the suggestions displayed on the app screen.

[0553] Step 11:

[0554] User: Selects an action based on the suggestion, for example, "Sell."

[0555] Step 12:

[0556] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[0557] Example 1

[0558] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0559] Currently, systems that appraise products and suggest appropriate disposal methods face challenges in accurately identifying product categories and detecting scratches and stains. In particular, the accuracy of image analysis is low, and appraisal results and disposal method suggestions can be inaccurate, making it difficult for users to obtain reliable information. Another problem is the poor usability of users when entering information into the system.

[0560] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0561] In this invention, the server includes: a means for a user to photograph or upload a product image; a means for transmitting the product image to the server; a means for the server to analyze the product image and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product image and adjust the market price; a means for the server to suggest the adjusted price and a disposal method to the user; a means for the user to select a next action based on the suggestion; a means for the server to use a generative AI model for image analysis; and a means for the user to provide a specific prompt sentence as input. This enables the user to accurately identify the product category and detect scratches or stains, and to receive a highly reliable appraisal result and a suggestion of an appropriate disposal method.

[0562] "User" refers to an individual or corporation that uses the System to take or upload product images and receive appraisals and disposal method suggestions.

[0563] "Product Image" refers to digital image data that a user photographs or uploads to represent an unwanted item.

[0564] "Server" refers to a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[0565] "Database" refers to an information management system that stores information such as product categories, market prices, and handling methods.

[0566] A "generative AI model" refers to a trained model that uses artificial intelligence technology to perform image analysis and identify product categories.

[0567] A "prompt sentence" refers to a sentence that allows a user to input specific instructions or information into a system.

[0568] A "convolutional neural network" is a type of deep learning algorithm used in image analysis, and refers to a technology that automatically extracts features from images.

[0569] "Image processing algorithm" refers to the computational procedures or programs used to detect flaws or stains from product images.

[0570] "Quoted Price" refers to the expected price in the market for a particular product category, as retrieved from a database.

[0571] A "proposal message" refers to a message generated by the server to present the adjusted price and the optimal disposal method to the user.

[0572] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system is composed of a server, terminals, and users. The program processing of this system is explained below.

[0573] The system hardware will use smartphones and tablets as user devices, a central computer system as a server, and an information management system as a database, while the software will include generative AI models and image processing algorithms.

[0574] First, the user uses a smartphone or tablet device to take a photo of the item they want to appraise or upload an existing image. This operation is performed via a dedicated application installed on the device.

[0575] The user device resizes the received image to an appropriate resolution and sends it to the server, which receives the image and uses a generative AI model (e.g., a convolutional neural network) to identify the product category. It then retrieves the market price of the identified product from a database.

[0576] The server then uses image processing algorithms to detect scratches and stains from the product images. Based on this information, the server adjusts the market price. Finally, the server generates a proposal message suggesting the adjusted price and the optimal disposal method (e.g., sell, keep, or destroy) and sends it to the user's device.

[0577] The user can check the suggestion message displayed on the terminal and select the next action based on the suggestion. An example of a specific prompt sentence is, "I uploaded a photo of an old jacket. Please check for defects and stains and let me know the market price."

[0578] Using this system, users can obtain accurate appraisal information for unwanted items and select the optimal action for efficient disposal. For example, if a user wants to have an old jacket appraised, they launch the app, take a photo, and upload it. The system analyzes the image and identifies the jacket's category and market price. It also detects a small stain on the chest, adjusts the price, and suggests selling the item. In this way, the present invention can provide users with highly accurate appraisals and convenient disposal method suggestions.

[0579] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0580] Step 1:

[0581] A user takes or uploads a product image.

[0582] Input: Product images taken by a user using a smartphone or tablet, or existing image data.

[0583] How it works: A user launches the app and either takes a photo of the item they want to appraise or selects and uploads an image from their gallery.

[0584] Output: The product image is displayed on the terminal and the button to proceed to the next step is activated.

[0585] Step 2:

[0586] The device preprocesses the uploaded image and sends it to the server.

[0587] Input: A user-uploaded image.

[0588] Data processing: The device resizes the received image to an appropriate resolution (e.g., 1024x768 pixels).

[0589] How it works: The device resizes the image and compresses it while preserving quality.

[0590] Output: The preprocessed image is sent to the server.

[0591] Step 3:

[0592] The server analyzes the image and identifies the product category.

[0593] Input: The preprocessed image received by the server.

[0594] Data computation: The server performs image analysis to identify product categories using a generative AI model (e.g., convolutional neural network).

[0595] How it works: A convolutional neural network extracts features from an image and matches them with information in a database.

[0596] Output: Identified product category (e.g., "Brand name jacket (2005 model)").

[0597] Step 4:

[0598] The server retrieves market prices from a database based on the identified product category.

[0599] Input: Identified product category.

[0600] Data acquisition: The server searches the database for the market price corresponding to the product category and acquires it.

[0601] How it works: The server generates a database query to retrieve detailed product information, including price quotes.

[0602] Output: Market price of the item (e.g., 5000 yen).

[0603] Step 5:

[0604] The server detects scratches and stains from product images and adjusts the market price.

[0605] Input: Product image and quote price.

[0606] Data processing: The server uses image processing algorithms to detect scratches and stains in the product images and adjust the market price.

[0607] How it works: Image analysis algorithms detect defects in images and recalculate prices based on that information.

[0608] Output: The adjusted price (e.g., $4000).

[0609] Step 6:

[0610] The server proposes the adjusted price and disposal method to the user.

[0611] Input: Adjusted price.

[0612] Data Generation: The server generates a proposal message containing the adjusted price and the optimal disposal method.

[0613] Behavior: The server creates a suggestion message that reads, "This jacket is recommended for sale. The current market price is about $40.00. Would you like to sell it?"

[0614] Output: The proposal message is sent to the terminal.

[0615] Step 7:

[0616] The user selects the next action based on the suggestions.

[0617] Input: The suggestion message displayed on the terminal.

[0618] Action: The user reviews the proposal and selects "Sell."

[0619] Output: The user's selection is communicated to the server, and the next action is performed.

[0620] (Application example 1)

[0621] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0622] Conventional systems that assess unwanted products and suggest disposal methods do not support frequently used ingredients or food delivery services. Furthermore, there are no tools that can quickly assess the freshness and condition of ingredients in the refrigerator and suggest optimal ways to use or dispose of them, leaving users without a way to easily reduce food waste. This results in a large amount of food waste within the home, which increases costs and puts a strain on the environment.

[0623] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0624] In this invention, the server includes: a means for a user to photograph or upload product images; a means for transmitting the product images to the server; a means for the server to analyze the product images and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product images and adjust the market price; a means for a user to photograph or upload an image of an ingredient; a means for transmitting the ingredient image to the server; a means for the server to analyze the ingredient image and identify a food category; a means for the server to obtain an expiration date and an optimal usage method from a database based on the identified food category; and a means for the server to detect the freshness and condition of the ingredient from the ingredient image and adjust the usage method. This allows a user to easily check the freshness of ingredients in the refrigerator and quickly find optimal usage and disposal methods to reduce waste.

[0625] A "user terminal" is a device that a user uses to take or upload images of products or ingredients.

[0626] The "server" is a central computer system that analyzes images of products and ingredients, identifies categories, adjusts prices, and detects freshness and condition.

[0627] A "database" is a system that stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[0628] "Product category" refers to the type or classification of a product identified by the server through analysis.

[0629] "Quoted Price" is the market price of the commodity obtained from the database.

[0630] "Scratches and stains" refer to external defects and stains detected from product images.

[0631] "Food image" is an image taken or uploaded by the user to understand the condition of the food in the refrigerator.

[0632] "Freshness" refers to the state of freshness of ingredients and storage conditions.

[0633] "Use by date" is the expiration date of the ingredient obtained from the database.

[0634] The "optimal use" is a method of use suggested by the server based on the freshness and condition of the ingredients.

[0635] The "adjusted price" is the adjusted market price calculated by the server based on the results of detecting scratches and stains.

[0636] The "suggestion result" is information about the adjusted price and how to use the ingredients that is displayed to the user.

[0637] An "action" is the next operation that the user selects based on the suggested results.

[0638] This invention is a system that analyzes the status of unwanted products and food items stored in a user's refrigerator and suggests optimal disposal and usage methods. The system is composed of a server, a user terminal, and a database.

[0639] System Configuration

[0640] User terminal

[0641] It is a device such as a smartphone or tablet that provides an interface for taking and uploading images of products and ingredients and operating applications.

[0642] server

[0643] It is a central computer system that analyzes images of products and ingredients, identifies categories, obtains market prices, detects scratches, dirt, freshness and condition, adjusts prices, and suggests ways to use and dispose of them. Specifically, it makes full use of convolutional neural networks (CNNs) using TensorFlow and Keras.

[0644] Database

[0645] This system stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[0646] Program processing flow

[0647] User terminal

[0648] The user takes or uploads an image of the product or ingredient.

[0649] The image is resized to the appropriate resolution and sent to the server.

[0650] server

[0651] The server receives the image and analyzes it using a convolutional neural network (CNN).

[0652] For products, the system identifies the category, obtains the market price, detects scratches and stains, and proposes an adjusted price and disposal method.

[0653] In the case of food ingredients, the system identifies the food category, obtains the expiration date and optimal usage method, detects the freshness and condition, and suggests how to use and dispose of the food.

[0654] Specific examples

[0655] Specific examples of product appraisal usage

[0656] 1. The user takes a photo of an old jacket (e.g., a 2005 model from Brand A), launches the app, and uploads the image.

[0657] 2. The device resizes the image and sends it to the server.

[0658] 3. The server analyzes the image, determines the market price of the jacket, detects scratches and stains, and calculates the adjusted price.

[0659] 4. The server generates a suggestion such as "We recommend selling this jacket. The market price is about 4,000 yen. Would you like to sell it?" and displays the suggestion result to the user.

[0660] 5. The user selects "Sell" based on the proposed results.

[0661] Food ingredient management example

[0662] 1. The user takes a photo of an apple in the refrigerator, launches the app and uploads the image.

[0663] 2. The device resizes the image and sends it to the server.

[0664] 3. The server analyzes the image, identifies the apple category, determines its freshness, and retrieves expiration date information from a database.

[0665] 4. The server generates a suggestion such as "This apple is fresh. We recommend eating it raw." and displays the suggestion result to the user.

[0666] 5. The user selects the next action to take based on the suggested results.

[0667] Prompt Sentence Examples

[0668] "Just take a photo of the food in your refrigerator and upload it to the app. The system will automatically determine the freshness of the food and tell you the best way to use it. For example, if you upload a photo of an apple, it will suggest, 'This apple is fresh. We recommend eating it raw.'"

[0669] As described above, the present invention provides a system that allows users to efficiently manage goods and ingredients by analyzing images of the goods and ingredients and proposing optimal disposal and usage methods.

[0670] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0671] Step 1:

[0672] The user takes or uploads an image of the product or ingredient.

[0673] Input: An image of a product or ingredient.

[0674] Specific operation: The user uses a user device such as a smartphone or tablet to take a photo of the target product or ingredient or upload an existing image.

[0675] Output: Image data captured or uploaded.

[0676] Step 2:

[0677] The terminal resizes the product or ingredient image to an appropriate resolution and transmits it to the server.

[0678] Input: Image data captured or uploaded.

[0679] Specific operation: The device resizes the image resolution to, for example, 1024x768 pixels and sends the image data to the server.

[0680] Output: The resized image data.

[0681] Step 3:

[0682] The server receives the images and analyzes them using a convolutional neural network (CNN).

[0683] Input: The resized image data.

[0684] Specific operation: An AI model (using TensorFlow and Keras) installed on the server reads the image and analyzes it using CNN to identify the product or ingredient category.

[0685] Output: Category information of products and ingredients.

[0686] Step 4:

[0687] The server retrieves the market price and expiration date from the database based on the identified category.

[0688] Input: Category information of product or ingredients.

[0689] Specific operation: The server accesses the database and retrieves the market price based on the identified product category, the expiration date based on the food ingredient category, and the latest information.

[0690] Output: Market price information for products or expiration date information for ingredients.

[0691] Step 5:

[0692] The server detects scratches and dirt from product images and detects freshness and condition from food ingredient images.

[0693] Input: Resized image data and category information.

[0694] How it works: The server uses image processing algorithms to detect scratches and dirt from product images, and freshness and condition from food images.

[0695] Output: Information on detected scratches and stains, freshness and condition.

[0696] Step 6:

[0697] Based on the detection results, the server will suggest product price adjustments and optimal ways to use and dispose of ingredients.

[0698] Input: Market price information or expiration date information, damage or stain information, freshness or condition information.

[0699] What it does: The server adjusts the market price based on the results of any damage or stains detected, and determines the best way to use or dispose of the ingredients.

[0700] Output: Adjusted price or recommendations on optimal usage and disposal methods.

[0701] Step 7:

[0702] The user selects the next action based on the suggestions.

[0703] Input: Proposal information.

[0704] Specific operation: The user checks the proposal results on their device and selects the next action (e.g., sell the product, use the ingredients for a delicious meal, or discard if unnecessary).

[0705] Output: The result of the selection of the next action to be taken.

[0706] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0707] This invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained in natural language below. Specific examples of use are also included.

[0708] System Configuration

[0709] 1. User Device

[0710] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[0711] 2. Server

[0712] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[0713] 3. Database

[0714] This is a system that stores information such as the market price, category, and handling method of a product.

[0715] 4. Emotion Engine

[0716] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[0717] Program processing flow (overview)

[0718] 1. Initial Setup and User Interface Display

[0719] User: Launch the app and take or upload a photo of the item you want appraised.

[0720] Terminal: Receives the image and sends it to the server.

[0721] 2. Image transmission and preprocessing

[0722] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[0723] Server: Receives images and prepares them for analysis.

[0724] 3. Image Recognition and Product Identification

[0725] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[0726] Server: Retrieve the market price for the identified item from the database.

[0727] 4. Detecting scratches and stains

[0728] Server: Uses image processing algorithms to detect scratches and stains from product images.

[0729] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[0730] 5. Emotion Analysis

[0731] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[0732] 6. Proposal generation based on final price and sentiment

[0733] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[0734] 7. Displaying the results

[0735] User: Check the proposed results.

[0736] User: Choose next action based on suggestions.

[0737] Specific examples

[0738] Specific use cases for product appraisal and emotion recognition

[0739] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[0740] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[0741] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[0742] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[0743] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[0744] 6. Server: Adjust price to 4000 based on the impact of this stain.

[0745] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[0746] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[0747] 9. User: Check the proposal results and select "Sell" or "Keep."

[0748] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[0749] The processing flow will be explained below.

[0750] Step 1:

[0751] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[0752] Step 2:

[0753] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[0754] Step 3:

[0755] On the device: The resized image data is sent to the server using an HTTP request.

[0756] Step 4:

[0757] Server: Decodes the received image data into an analyzable format.

[0758] Step 5:

[0759] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[0760] Step 6:

[0761] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[0762] Step 7:

[0763] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[0764] Step 8:

[0765] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[0766] Step 9:

[0767] Emotion Engine: Analyzes facial expressions in real time from the user's camera to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[0768] Step 10:

[0769] Server: Based on the user's emotional information obtained from the emotion engine, the server proposes an adjusted price and the optimal disposal method (sell, keep, or destroy). For example, if the user shows signs of being emotionally reluctant to sell, the server will suggest keeping the item.

[0770] Step 11:

[0771] Server: Generates a suggestion message to the user, such as "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" or "Or, consider keeping it," and displays it on the screen.

[0772] Step 12:

[0773] User: Check the proposed results.

[0774] Step 13:

[0775] User: Selects an action based on the suggestion, for example, "sell" or "keep."

[0776] Step 14:

[0777] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[0778] Example 2

[0779] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0780] Conventional unwanted item appraisal systems have the problem that it is difficult to accurately evaluate the condition of products and do not provide personalized suggestions that take the user's emotions into consideration, resulting in low user satisfaction. This invention aims to achieve more accurate appraisals and suggestions that result in higher user satisfaction by combining product image analysis and an emotion engine.

[0781] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing a product image and identifying a product category, a means for acquiring a market price from a database based on the identified product category, a means for detecting scratches or stains from the product image and adjusting the market price, a means for analyzing the user's emotional state using an emotion engine, and a means for adjusting the proposal content based on the user's emotional state. This enables more accurate appraisals and personalized proposals that take the user's emotions into consideration.

[0782] "Product image" refers to a photograph or image data of the product for which the user wishes to have it appraised.

[0783] A "server" is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and generates suggestions in cooperation with an emotion engine.

[0784] "Product category" is information that indicates the type or classification of a specific product, and by identifying it, it becomes possible to obtain the market price.

[0785] "Quoted Price" refers to the prevailing price in the market for a particular commodity, and is obtained from a database.

[0786] A "database" refers to an information system that stores information such as the market price of a product, category information, and handling instructions.

[0787] "Scratches and stains" refers to physical damage or stains on the surface of the product, which are factors that affect the value of the product.

[0788] An "emotion engine" refers to an algorithm or system that analyzes a user's facial expressions and voice to recognize their emotional state.

[0789] "Adjusting the proposal" refers to optimizing the proposal for how to dispose of the product and the final price based on the user's emotional state.

[0790] An "action" refers to a decision or action (e.g., sell, keep, or discard) that a user makes based on a suggestion from the server.

[0791] The present invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained below in natural language.

[0792] System Configuration

[0793] 1. User Device

[0794] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[0795] 2. Server

[0796] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[0797] 3. Database

[0798] This is a system that stores information such as the market price, category, and handling method of a product.

[0799] 4. Emotion Engine

[0800] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[0801] Program Processing Overview

[0802] The program operates as follows.

[0803] 1. Initial Setup and User Interface Display

[0804] User: Launch the app and take or upload a photo of the item you want appraised.

[0805] Terminal: Receives the image and sends it to the server.

[0806] 2. Image transmission and preprocessing

[0807] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[0808] Server: Receives images and prepares them for analysis.

[0809] 3. Image Recognition and Product Identification

[0810] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[0811] Server: Retrieve the market price for the identified item from the database.

[0812] 4. Detecting scratches and stains

[0813] Server: Uses image processing algorithms to detect scratches and stains from product images.

[0814] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[0815] 5. Emotion Analysis

[0816] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[0817] 6. Proposal generation based on final price and sentiment

[0818] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[0819] 7. Displaying the results

[0820] User: Check the proposed results.

[0821] User: Choose next action based on suggestions.

[0822] Specific examples

[0823] Specific use cases for product appraisal and emotion recognition

[0824] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[0825] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[0826] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[0827] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[0828] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[0829] 6. Server: Adjust price to 4000 based on the impact of this stain.

[0830] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[0831] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[0832] 9. User: Check the proposal results and select "Sell" or "Keep."

[0833] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[0834] Specific prompt examples

[0835] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[0836] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0837] Step 1:

[0838] Initial Setup and User Interface Display

[0839] User: Launches the app and takes a photo of the item they wish to have appraised. The input is a photo of the item. Specifically, the user taps the smartphone screen to launch the app, selects camera mode, and takes a photo. The output is the captured photo data.

[0840] Step 2:

[0841] Image transmission and preprocessing

[0842] Terminal: Resizes a captured photo to a specified resolution (for example, 1024x768 pixels). The input is the captured photo data, which undergoes resolution conversion. Specifically, the application changes the image size internally. The output is the resized image data.

[0843] Device: Sends the resized image to the server. The input is the resized image data, which goes through a process of being sent to the server. Specifically, the device uploads the image to the server using an HTTP request. The output is the image data sent to the server.

[0844] Step 3:

[0845] Image recognition and product identification

[0846] Server: Prepares the received image for analysis. The input is the transmitted image data, which is converted into an internal format so that it can be analyzed. The output is image data that can be analyzed.

[0847] Server: Identifies product categories from images using a convolutional neural network (CNN). The input is analyzable image data, features are extracted using a CNN model, and product categories are identified using a classification model. Specifically, the CNN is executed to identify categories such as "2005 model jackets from a certain brand." The output is the identified product category information.

[0848] Server: Retrieves the market price from the database based on the identified product category. The input is the product category information, and a database query is made based on this. The output is the market price (e.g., 5000 yen).

[0849] Step 4:

[0850] Detecting scratches and stains

[0851] Server: Uses image processing algorithms to detect scratches and stains from product images. The input is analyzable image data, and the image processing algorithm identifies areas of physical damage or stains. Specifically, the algorithm detects stains on the chest of a jacket. The output is information about the detected scratches and stains (area and degree).

[0852] Server: Adjusts the market price based on the detected scratches and stains. The input is the scratch and stain information and the market price, and a comprehensive evaluation is performed to calculate the final price. Specifically, the market price is adjusted to 4000 yen due to the influence of stains. The output is the final price after adjustment.

[0853] Step 5:

[0854] Emotion Analysis

[0855] Emotion engine: Analyzes facial expressions from the user's camera footage in real time to identify the user's emotional state. The input is the user's camera footage, to which an expression analysis algorithm is applied. Specifically, it analyzes the user's facial expressions when viewing the assessment results and determines whether the user is expressing happiness, sadness, surprise, etc. The output is information about the user's emotional state.

[0856] Emotion engine: Analyzes the user's voice to enhance emotional information. The input is the user's voice data, and the voice analysis algorithm supplements the emotional state. The output is integrated emotional state information.

[0857] Step 6:

[0858] Generate offers based on final price and sentiment

[0859] Server: Generates individual proposals based on the adjusted price and the user's emotional information. The input is the final price and the integrated emotional state information. Specifically, the server generates a proposal based on the adjusted price (4000 yen) and the emotional information, such as "We recommend selling this jacket, and the current market price is about 4000 yen. Would you like to sell it?" If the emotion is negative, it also presents an alternative suggestion such as "One option is to consider keeping it." The output is the generated proposal.

[0860] Step 7:

[0861] Displaying the results

[0862] User: Checks the proposal results from the server through the application. The input is the proposal content from the server. The specific operation is that the proposal results are displayed on the smartphone screen. The output is the confirmed proposal results.

[0863] User: Selects the next action based on the suggestion. The input is the suggestion result, and based on that, the user selects an action such as "sell" or "keep." Specific actions are performed by the user tapping an option on the screen. The output is the selected action.

[0864] (Example prompt)

[0865] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[0866] (Application example 2)

[0867] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0868] Conventional food delivery services have been unable to provide personalized menu suggestions or special offers based on the user's emotional state, which makes it difficult for users to receive appropriate service that matches their mood and emotions at the time, potentially resulting in lower satisfaction.

[0869] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for identifying the user's emotion and adjusting the proposal content based on the identification result, means for analyzing product images and identifying the product category, and means for retrieving market prices from a database based on the identified product category. This makes it possible to provide personalized menu suggestions and special offers based on the user's emotional state.

[0870] "Means for users to take or upload product images" refers to devices or software that provide a function for users to take product images and upload them to a server through an application.

[0871] The "means for transmitting the product image to the server" is a communication interface for transmitting the photographed or uploaded product image to the server via the Internet.

[0872] "Means for the server to analyze the product image and identify the product category" refers to a process in which the server uses an image analysis algorithm on the product image received to identify the category to which the product belongs.

[0873] The "means for the server to obtain market price information from the database based on the identified product category" is a process in which the server obtains market price information that matches the identified product category from the database.

[0874] "Means for the server to detect scratches or stains from product images and adjust the market price" refers to a process in which the server analyzes product images, detects the presence or absence of scratches or stains and their extent, and adjusts the market price.

[0875] The "means for the server to propose the adjusted price and disposal method to the user" is a process in which the server proposes to the user, along with the adjusted price information, a disposal method such as whether to sell, keep, or discard the product.

[0876] The "means for the user to select the next action based on the proposal" is an interface that allows the user to receive a proposal from the server and select the next action to be taken (sell, keep, discard, etc.).

[0877] "Means for identifying the user's emotions and adjusting the content of suggestions based on the identification results" refers to a process in which an emotion engine is used to identify emotions from the user's facial expressions and voice, and the content of suggestions is personalized based on the results.

[0878] The present invention relates to a personalized menu recommendation system that utilizes an emotion engine to improve user experience in food delivery services. Specific embodiments for implementing this system will be described below.

[0879] 1. System Configuration

[0880] User terminal

[0881] A device used by a user, such as a smartphone or tablet, that includes a camera and microphone, allowing the user to register emotions through facial expressions and voice, and has an application installed on it that provides an interface for launching the application and placing a delivery order.

[0882] server

[0883] This is a central computer system for analyzing product images, retrieving market prices from a database, detecting scratches and stains, adjusting prices, suggesting disposal methods, and making suggestions based on user emotion recognition. This includes an emotion engine, image analysis algorithms, and database access.

[0884] Database

[0885] This is a system that stores information such as the market price, category, and handling method of a product.

[0886] Emotion Engine

[0887] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[0888] 2. Hardware and software used

[0889] Hardware

[0890] Smartphones and tablets (with cameras and microphones)

[0891] software

[0892] Emotion engine: OpenCV (facial expression analysis), Google Speech-to-Text API (voice analysis)

[0893] Database: MongoDB (Management of menu / emotion pairing data)

[0894] Server: AWS Lambda (serverless environment), Amazon RDS (data storage)

[0895] 3. Data processing and calculation

[0896] User terminal

[0897] Emotional data is collected by users showing facial expressions to the camera and inputting voice data through a microphone, and this data is sent to a server via the application.

[0898] Emotion analysis

[0899] The facial expression data received by the server is analyzed by OpenCV, and the voice data is converted to text using the Google Speech-to-Text API, which allows for specific identification of the user's emotional state.

[0900] Database Access and Proposal Generation

[0901] The emotion engine retrieves appropriate menus from the database according to the user's emotional state, for example, suggesting menus that match a specific emotional state (e.g., if you are tired, suggesting menus that will cheer you up).

[0902] Suggestions for users

[0903] The server sends the menu suggestions along with the adjusted price information to the user's device, which is then displayed on the application to help the user choose the best course of action.

[0904] 4. Specific Examples

[0905] Specific examples of menu suggestions

[0906] 1. The user smiles at the camera.

[0907] 2. Saying "I'm tired today" on the microphone.

[0908] 3. OpenCV analyzes whether or not a smile is present and scores the degree of joy.

[0909] 4. The Google Speech-to-Text API converts "I'm tired today" into text and feeds it back into the emotion engine.

[0910] 5. From the database, a "nutritious chicken salad bowl" is suggested as a menu suitable for the "tired" and "happy" scores.

[0911] 6. The suggested menu is displayed within the app and the user taps the "Order" button.

[0912] Examples of prompt statements

[0913] This prompt is used for the generative AI model:

[0914] Suggest cheer-up foods when the user says "I'm feeling down today" and smiles at the camera. Describe how your system personalizes the suggestions and reassessss whether the user is satisfied with them. Use an emotion engine to analyze the user's emotional feedback in real time and adjust the suggestions accordingly. Use smartphone hardware and simulate the processing flow using OpenCV and the Google Speech-To-Text API.

[0915] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0916] Step 1:

[0917] The user launches the food delivery app. The user shows their facial expression in front of the camera and inputs voice using the microphone. The input data is the user's facial image and voice input. The facial image and voice input are saved on the device and prepared for transmission to the server.

[0918] Step 2:

[0919] The device sends the facial expression images and voice input data it has saved to the server. The data input here are facial expression images and voice files, and communication is performed to send them to the server. This allows the server to begin analyzing the user's emotional data.

[0920] Step 3:

[0921] The server analyzes the received facial image using OpenCV and identifies the emotion from the user's facial expression. The input is the facial image, and the output is an emotion score (e.g., joy, sadness, surprise, etc.). The OpenCV algorithm extracts facial features and scores the emotion based on them.

[0922] Step 4:

[0923] The server converts the received voice data into a string using the Google Speech-to-Text API. The input is voice data and the output is text. Speech recognition technology converts the voice data into text data and provides additional information about the user's emotions.

[0924] Step 5:

[0925] The server combines the results of facial expression analysis and voice analysis to determine the overall emotional state. The input is the emotion score and text data, and the output is the overall emotional state (e.g., tired but happy). The emotion engine combines these data to determine the emotional state with greater accuracy.

[0926] Step 6:

[0927] The server selects an appropriate menu from a database based on the overall emotional state. The input is the emotional state, and the output is the corresponding menu information (e.g., candidates for energizing foods). A database search is performed to obtain menus that match the emotion.

[0928] Step 7:

[0929] The server sends the selected menu information to the terminal. The input here is the menu information, and the output is a menu suggestion that is displayed on the user's terminal. The suggested menu and the reason for it are displayed in the user application.

[0930] Step 8:

[0931] The user reviews and selects a menu suggestion from the server. The input here is the suggested menu, and the output is the user's choice (e.g., confirming the order). The user browses the suggested menu on the app and taps a button to confirm the order.

[0932] Step 9:

[0933] The server stores the user's selections and processes the order. The input is the user's selections and the output is the actual order data. The server stores the order information and issues the order to the food delivery service.

[0934] Step 10:

[0935] After the server places an order, it collects user feedback and adds it to the training data to improve the accuracy of the emotion engine. The input here is user feedback, and the output is an updated emotion data model. User feedback is collected periodically and the emotion engine model is updated.

[0936] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0937] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0938] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0939] [Third embodiment]

[0940] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0941] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0942] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0943] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0944] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0945] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0946] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0947] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0948] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0949] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0950] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0951] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0952] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system consists of a server, a terminal, and a user. The program processing of this system is explained below in natural language. Specific examples of use are also included.

[0953] System Configuration

[0954] 1. User Device

[0955] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[0956] 2. Server

[0957] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[0958] 3. Database

[0959] This is a system that stores information such as the market price, category, and handling method of a product.

[0960] Program processing flow (overview)

[0961] 1. Initial Setup and User Interface Display

[0962] User: Launches the app and takes or uploads a product image.

[0963] Terminal: Accepts images and sends them to the server.

[0964] 2. Image transmission and preprocessing

[0965] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[0966] Server: Receives images and prepares them for analysis.

[0967] 3. Image Recognition and Product Identification

[0968] Server: Identifies product categories using AI techniques (e.g., convolutional neural networks).

[0969] Server: Retrieve the market price for the identified item from the database.

[0970] 4. Detecting scratches and stains

[0971] Server: Uses image processing algorithms to detect scratches and stains from product images.

[0972] Server: Adjusts the market price based on the degree of damage or dirt detected.

[0973] 5. Generate final price and proposal

[0974] Server: Proposes the adjusted price and the optimal disposal method (sell, keep, or destroy) to the user.

[0975] 6. Displaying the results

[0976] User: Check the proposed results.

[0977] User: Choose next action based on suggestions.

[0978] Specific examples

[0979] Specific examples of product appraisal usage

[0980] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[0981] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[0982] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[0983] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[0984] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[0985] 6. Server: Adjust price to 4000 based on the impact of this stain.

[0986] 7. Server: Generate a suggestion: "This jacket is worth selling. The current market price is about 4000 yen. Would you like to sell it?"

[0987] 8. User: Check the proposal results and select "Sell."

[0988] This system allows users to obtain accurate valuation information for unwanted items and choose the best course of action to dispose of them efficiently.

[0989] The processing flow will be explained below.

[0990] Step 1:

[0991] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[0992] Step 2:

[0993] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[0994] Step 3:

[0995] On the device: The resized image data is sent to the server using an HTTP request.

[0996] Step 4:

[0997] Server: Decodes the received image data into an analyzable format.

[0998] Step 5:

[0999] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[1000] Step 6:

[1001] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[1002] Step 7:

[1003] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[1004] Step 8:

[1005] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[1006] Step 9:

[1007] Server: Determines the adjusted price and the proposed disposal method (sell, keep, or destroy), and generates a message to the user. For example, it creates a message saying, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?"

[1008] Step 10:

[1009] User: Check the suggestions displayed on the app screen.

[1010] Step 11:

[1011] User: Selects an action based on the suggestion, for example, "Sell."

[1012] Step 12:

[1013] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[1014] Example 1

[1015] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1016] Currently, systems that appraise products and suggest appropriate disposal methods face challenges in accurately identifying product categories and detecting scratches and stains. In particular, the accuracy of image analysis is low, and appraisal results and disposal method suggestions can be inaccurate, making it difficult for users to obtain reliable information. Another problem is the poor usability of users when entering information into the system.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1018] In this invention, the server includes: a means for a user to photograph or upload a product image; a means for transmitting the product image to the server; a means for the server to analyze the product image and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product image and adjust the market price; a means for the server to suggest the adjusted price and a disposal method to the user; a means for the user to select a next action based on the suggestion; a means for the server to use a generative AI model for image analysis; and a means for the user to provide a specific prompt sentence as input. This enables the user to accurately identify the product category and detect scratches or stains, and to receive a highly reliable appraisal result and a suggestion of an appropriate disposal method.

[1019] "User" refers to an individual or corporation that uses the System to take or upload product images and receive appraisals and disposal method suggestions.

[1020] "Product Image" refers to digital image data that a user photographs or uploads to represent an unwanted item.

[1021] "Server" refers to a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[1022] "Database" refers to an information management system that stores information such as product categories, market prices, and handling methods.

[1023] A "generative AI model" refers to a trained model that uses artificial intelligence technology to perform image analysis and identify product categories.

[1024] A "prompt sentence" refers to a sentence that allows a user to input specific instructions or information into a system.

[1025] A "convolutional neural network" is a type of deep learning algorithm used in image analysis, and refers to a technology that automatically extracts features from images.

[1026] "Image processing algorithm" refers to the computational procedures or programs used to detect flaws or stains from product images.

[1027] "Quoted Price" refers to the expected price in the market for a particular product category, as retrieved from a database.

[1028] A "proposal message" refers to a message generated by the server to present the adjusted price and the optimal disposal method to the user.

[1029] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system is composed of a server, terminals, and users. The program processing of this system is explained below.

[1030] The system hardware will use smartphones and tablets as user devices, a central computer system as a server, and an information management system as a database, while the software will include generative AI models and image processing algorithms.

[1031] First, the user uses a smartphone or tablet device to take a photo of the item they want to appraise or upload an existing image. This operation is performed via a dedicated application installed on the device.

[1032] The user device resizes the received image to an appropriate resolution and sends it to the server, which receives the image and uses a generative AI model (e.g., a convolutional neural network) to identify the product category. It then retrieves the market price of the identified product from a database.

[1033] The server then uses image processing algorithms to detect scratches and stains from the product images. Based on this information, the server adjusts the market price. Finally, the server generates a proposal message suggesting the adjusted price and the optimal disposal method (e.g., sell, keep, or destroy) and sends it to the user's device.

[1034] The user can check the suggestion message displayed on the terminal and select the next action based on the suggestion. An example of a specific prompt sentence is, "I uploaded a photo of an old jacket. Please check for defects and stains and let me know the market price."

[1035] Using this system, users can obtain accurate appraisal information for unwanted items and select the optimal action for efficient disposal. For example, if a user wants to have an old jacket appraised, they launch the app, take a photo, and upload it. The system analyzes the image and identifies the jacket's category and market price. It also detects a small stain on the chest, adjusts the price, and suggests selling the item. In this way, the present invention can provide users with highly accurate appraisals and convenient disposal method suggestions.

[1036] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1037] Step 1:

[1038] A user takes or uploads a product image.

[1039] Input: Product images taken by a user using a smartphone or tablet, or existing image data.

[1040] How it works: A user launches the app and either takes a photo of the item they want to appraise or selects and uploads an image from their gallery.

[1041] Output: The product image is displayed on the terminal and the button to proceed to the next step is activated.

[1042] Step 2:

[1043] The device preprocesses the uploaded image and sends it to the server.

[1044] Input: A user-uploaded image.

[1045] Data processing: The device resizes the received image to an appropriate resolution (e.g., 1024x768 pixels).

[1046] How it works: The device resizes the image and compresses it while preserving quality.

[1047] Output: The preprocessed image is sent to the server.

[1048] Step 3:

[1049] The server analyzes the image and identifies the product category.

[1050] Input: The preprocessed image received by the server.

[1051] Data computation: The server performs image analysis to identify product categories using a generative AI model (e.g., convolutional neural network).

[1052] How it works: A convolutional neural network extracts features from an image and matches them with information in a database.

[1053] Output: Identified product category (e.g., "Brand name jacket (2005 model)").

[1054] Step 4:

[1055] The server retrieves market prices from a database based on the identified product category.

[1056] Input: Identified product category.

[1057] Data acquisition: The server searches the database for the market price corresponding to the product category and acquires it.

[1058] How it works: The server generates a database query to retrieve detailed product information, including price quotes.

[1059] Output: Market price of the item (e.g., 5000 yen).

[1060] Step 5:

[1061] The server detects scratches and stains from product images and adjusts the market price.

[1062] Input: Product image and quote price.

[1063] Data processing: The server uses image processing algorithms to detect scratches and stains in the product images and adjust the market price.

[1064] How it works: Image analysis algorithms detect defects in images and recalculate prices based on that information.

[1065] Output: The adjusted price (e.g., $4000).

[1066] Step 6:

[1067] The server proposes the adjusted price and disposal method to the user.

[1068] Input: Adjusted price.

[1069] Data Generation: The server generates a proposal message containing the adjusted price and the optimal disposal method.

[1070] Behavior: The server creates a suggestion message that reads, "This jacket is recommended for sale. The current market price is about $40.00. Would you like to sell it?"

[1071] Output: The proposal message is sent to the terminal.

[1072] Step 7:

[1073] The user selects the next action based on the suggestions.

[1074] Input: The suggestion message displayed on the terminal.

[1075] Action: The user reviews the proposal and selects "Sell."

[1076] Output: The user's selection is communicated to the server, and the next action is performed.

[1077] (Application example 1)

[1078] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1079] Conventional systems that assess unwanted products and suggest disposal methods do not support frequently used ingredients or food delivery services. Furthermore, there are no tools that can quickly assess the freshness and condition of ingredients in the refrigerator and suggest optimal ways to use or dispose of them, leaving users without a way to easily reduce food waste. This results in a large amount of food waste within the home, which increases costs and puts a strain on the environment.

[1080] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1081] In this invention, the server includes: a means for a user to photograph or upload product images; a means for transmitting the product images to the server; a means for the server to analyze the product images and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product images and adjust the market price; a means for a user to photograph or upload an image of an ingredient; a means for transmitting the ingredient image to the server; a means for the server to analyze the ingredient image and identify a food category; a means for the server to obtain an expiration date and an optimal usage method from a database based on the identified food category; and a means for the server to detect the freshness and condition of the ingredient from the ingredient image and adjust the usage method. This allows a user to easily check the freshness of ingredients in the refrigerator and quickly find optimal usage and disposal methods to reduce waste.

[1082] A "user terminal" is a device that a user uses to take or upload images of products or ingredients.

[1083] The "server" is a central computer system that analyzes images of products and ingredients, identifies categories, adjusts prices, and detects freshness and condition.

[1084] A "database" is a system that stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[1085] "Product category" refers to the type or classification of a product identified by the server through analysis.

[1086] "Quoted Price" is the market price of the commodity obtained from the database.

[1087] "Scratches and stains" refer to external defects and stains detected from product images.

[1088] "Food image" is an image taken or uploaded by the user to understand the condition of the food in the refrigerator.

[1089] "Freshness" refers to the state of freshness of ingredients and storage conditions.

[1090] "Use by date" is the expiration date of the ingredient obtained from the database.

[1091] The "optimal use" is a method of use suggested by the server based on the freshness and condition of the ingredients.

[1092] The "adjusted price" is the adjusted market price calculated by the server based on the results of detecting scratches and stains.

[1093] The "suggestion result" is information about the adjusted price and how to use the ingredients that is displayed to the user.

[1094] An "action" is the next operation that the user selects based on the suggested results.

[1095] This invention is a system that analyzes the status of unwanted products and food items stored in a user's refrigerator and suggests optimal disposal and usage methods. The system is composed of a server, a user terminal, and a database.

[1096] System Configuration

[1097] User terminal

[1098] It is a device such as a smartphone or tablet that provides an interface for taking and uploading images of products and ingredients and operating applications.

[1099] server

[1100] It is a central computer system that analyzes images of products and ingredients, identifies categories, obtains market prices, detects scratches, dirt, freshness and condition, adjusts prices, and suggests ways to use and dispose of them. Specifically, it makes full use of convolutional neural networks (CNNs) using TensorFlow and Keras.

[1101] Database

[1102] This system stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[1103] Program processing flow

[1104] User terminal

[1105] The user takes or uploads an image of the product or ingredient.

[1106] The image is resized to the appropriate resolution and sent to the server.

[1107] server

[1108] The server receives the image and analyzes it using a convolutional neural network (CNN).

[1109] For products, the system identifies the category, obtains the market price, detects scratches and stains, and proposes an adjusted price and disposal method.

[1110] In the case of food ingredients, the system identifies the food category, obtains the expiration date and optimal usage method, detects the freshness and condition, and suggests how to use and dispose of the food.

[1111] Specific examples

[1112] Specific examples of product appraisal usage

[1113] 1. The user takes a photo of an old jacket (e.g., a 2005 model from Brand A), launches the app, and uploads the image.

[1114] 2. The device resizes the image and sends it to the server.

[1115] 3. The server analyzes the image, determines the market price of the jacket, detects scratches and stains, and calculates the adjusted price.

[1116] 4. The server generates a suggestion such as "We recommend selling this jacket. The market price is about 4,000 yen. Would you like to sell it?" and displays the suggestion result to the user.

[1117] 5. The user selects "Sell" based on the proposed results.

[1118] Food ingredient management example

[1119] 1. The user takes a photo of an apple in the refrigerator, launches the app and uploads the image.

[1120] 2. The device resizes the image and sends it to the server.

[1121] 3. The server analyzes the image, identifies the apple category, determines its freshness, and retrieves expiration date information from a database.

[1122] 4. The server generates a suggestion such as "This apple is fresh. We recommend eating it raw." and displays the suggestion result to the user.

[1123] 5. The user selects the next action to take based on the suggested results.

[1124] Prompt Sentence Examples

[1125] "Just take a photo of the food in your refrigerator and upload it to the app. The system will automatically determine the freshness of the food and tell you the best way to use it. For example, if you upload a photo of an apple, it will suggest, 'This apple is fresh. We recommend eating it raw.'"

[1126] As described above, the present invention provides a system that allows users to efficiently manage goods and ingredients by analyzing images of the goods and ingredients and proposing optimal disposal and usage methods.

[1127] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1128] Step 1:

[1129] The user takes or uploads an image of the product or ingredient.

[1130] Input: An image of a product or ingredient.

[1131] Specific operation: The user uses a user device such as a smartphone or tablet to take a photo of the target product or ingredient or upload an existing image.

[1132] Output: Image data captured or uploaded.

[1133] Step 2:

[1134] The terminal resizes the product or ingredient image to an appropriate resolution and transmits it to the server.

[1135] Input: Image data captured or uploaded.

[1136] Specific operation: The device resizes the image resolution to, for example, 1024x768 pixels and sends the image data to the server.

[1137] Output: The resized image data.

[1138] Step 3:

[1139] The server receives the images and analyzes them using a convolutional neural network (CNN).

[1140] Input: The resized image data.

[1141] Specific operation: An AI model (using TensorFlow and Keras) installed on the server reads the image and analyzes it using CNN to identify the product or ingredient category.

[1142] Output: Category information of products and ingredients.

[1143] Step 4:

[1144] The server retrieves the market price and expiration date from the database based on the identified category.

[1145] Input: Category information of product or ingredients.

[1146] Specific operation: The server accesses the database and retrieves the market price based on the identified product category, the expiration date based on the food ingredient category, and the latest information.

[1147] Output: Market price information for products or expiration date information for ingredients.

[1148] Step 5:

[1149] The server detects scratches and dirt from product images and detects freshness and condition from food ingredient images.

[1150] Input: Resized image data and category information.

[1151] How it works: The server uses image processing algorithms to detect scratches and dirt from product images, and freshness and condition from food images.

[1152] Output: Information on detected scratches and stains, freshness and condition.

[1153] Step 6:

[1154] Based on the detection results, the server will suggest product price adjustments and optimal ways to use and dispose of ingredients.

[1155] Input: Market price information or expiration date information, damage or stain information, freshness or condition information.

[1156] What it does: The server adjusts the market price based on the results of any damage or stains detected, and determines the best way to use or dispose of the ingredients.

[1157] Output: Adjusted price or recommendations on optimal usage and disposal methods.

[1158] Step 7:

[1159] The user selects the next action based on the suggestions.

[1160] Input: Proposal information.

[1161] Specific operation: The user checks the proposal results on their device and selects the next action (e.g., sell the product, use the ingredients for a delicious meal, or discard if unnecessary).

[1162] Output: The result of the selection of the next action to be taken.

[1163] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1164] This invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained in natural language below. Specific examples of use are also included.

[1165] System Configuration

[1166] 1. User Device

[1167] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[1168] 2. Server

[1169] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[1170] 3. Database

[1171] This is a system that stores information such as the market price, category, and handling method of a product.

[1172] 4. Emotion Engine

[1173] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[1174] Program processing flow (overview)

[1175] 1. Initial Setup and User Interface Display

[1176] User: Launch the app and take or upload a photo of the item you want appraised.

[1177] Terminal: Receives the image and sends it to the server.

[1178] 2. Image transmission and preprocessing

[1179] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[1180] Server: Receives images and prepares them for analysis.

[1181] 3. Image Recognition and Product Identification

[1182] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[1183] Server: Retrieve the market price for the identified item from the database.

[1184] 4. Detecting scratches and stains

[1185] Server: Uses image processing algorithms to detect scratches and stains from product images.

[1186] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[1187] 5. Emotion Analysis

[1188] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[1189] 6. Proposal generation based on final price and sentiment

[1190] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[1191] 7. Displaying the results

[1192] User: Check the proposed results.

[1193] User: Choose next action based on suggestions.

[1194] Specific examples

[1195] Specific use cases for product appraisal and emotion recognition

[1196] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[1197] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[1198] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[1199] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[1200] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[1201] 6. Server: Adjust price to 4000 based on the impact of this stain.

[1202] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[1203] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[1204] 9. User: Check the proposal results and select "Sell" or "Keep."

[1205] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[1206] The processing flow will be explained below.

[1207] Step 1:

[1208] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[1209] Step 2:

[1210] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[1211] Step 3:

[1212] On the device: The resized image data is sent to the server using an HTTP request.

[1213] Step 4:

[1214] Server: Decodes the received image data into an analyzable format.

[1215] Step 5:

[1216] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[1217] Step 6:

[1218] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[1219] Step 7:

[1220] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[1221] Step 8:

[1222] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[1223] Step 9:

[1224] Emotion Engine: Analyzes facial expressions in real time from the user's camera to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[1225] Step 10:

[1226] Server: Based on the user's emotional information obtained from the emotion engine, the server proposes an adjusted price and the optimal disposal method (sell, keep, or destroy). For example, if the user shows signs of being emotionally reluctant to sell, the server will suggest keeping the item.

[1227] Step 11:

[1228] Server: Generates a suggestion message to the user, such as "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" or "Or, consider keeping it," and displays it on the screen.

[1229] Step 12:

[1230] User: Check the proposed results.

[1231] Step 13:

[1232] User: Selects an action based on the suggestion, for example, "sell" or "keep."

[1233] Step 14:

[1234] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[1235] Example 2

[1236] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1237] Conventional unwanted item appraisal systems have the problem that it is difficult to accurately evaluate the condition of products and do not provide personalized suggestions that take the user's emotions into consideration, resulting in low user satisfaction. This invention aims to achieve more accurate appraisals and suggestions that result in higher user satisfaction by combining product image analysis and an emotion engine.

[1238] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing a product image and identifying a product category, a means for acquiring a market price from a database based on the identified product category, a means for detecting scratches or stains from the product image and adjusting the market price, a means for analyzing the user's emotional state using an emotion engine, and a means for adjusting the proposal content based on the user's emotional state. This enables more accurate appraisals and personalized proposals that take the user's emotions into consideration.

[1239] "Product image" refers to a photograph or image data of the product for which the user wishes to have it appraised.

[1240] A "server" is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and generates suggestions in cooperation with an emotion engine.

[1241] "Product category" is information that indicates the type or classification of a specific product, and by identifying it, it becomes possible to obtain the market price.

[1242] "Quoted Price" refers to the prevailing price in the market for a particular commodity, and is obtained from a database.

[1243] A "database" refers to an information system that stores information such as the market price of a product, category information, and handling instructions.

[1244] "Scratches and stains" refers to physical damage or stains on the surface of the product, which are factors that affect the value of the product.

[1245] An "emotion engine" refers to an algorithm or system that analyzes a user's facial expressions and voice to recognize their emotional state.

[1246] "Adjusting the proposal" refers to optimizing the proposal for how to dispose of the product and the final price based on the user's emotional state.

[1247] An "action" refers to a decision or action (e.g., sell, keep, or discard) that a user makes based on a suggestion from the server.

[1248] The present invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained below in natural language.

[1249] System Configuration

[1250] 1. User Device

[1251] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[1252] 2. Server

[1253] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[1254] 3. Database

[1255] This is a system that stores information such as the market price, category, and handling method of a product.

[1256] 4. Emotion Engine

[1257] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[1258] Program Processing Overview

[1259] The program operates as follows.

[1260] 1. Initial Setup and User Interface Display

[1261] User: Launch the app and take or upload a photo of the item you want appraised.

[1262] Terminal: Receives the image and sends it to the server.

[1263] 2. Image transmission and preprocessing

[1264] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[1265] Server: Receives images and prepares them for analysis.

[1266] 3. Image Recognition and Product Identification

[1267] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[1268] Server: Retrieve the market price for the identified item from the database.

[1269] 4. Detecting scratches and stains

[1270] Server: Uses image processing algorithms to detect scratches and stains from product images.

[1271] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[1272] 5. Emotion Analysis

[1273] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[1274] 6. Proposal generation based on final price and sentiment

[1275] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[1276] 7. Displaying the results

[1277] User: Check the proposed results.

[1278] User: Choose next action based on suggestions.

[1279] Specific examples

[1280] Specific use cases for product appraisal and emotion recognition

[1281] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[1282] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[1283] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[1284] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[1285] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[1286] 6. Server: Adjust price to 4000 based on the impact of this stain.

[1287] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[1288] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[1289] 9. User: Check the proposal results and select "Sell" or "Keep."

[1290] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[1291] Specific prompt examples

[1292] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[1293] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1294] Step 1:

[1295] Initial Setup and User Interface Display

[1296] User: Launches the app and takes a photo of the item they wish to have appraised. The input is a photo of the item. Specifically, the user taps the smartphone screen to launch the app, selects camera mode, and takes a photo. The output is the captured photo data.

[1297] Step 2:

[1298] Image transmission and preprocessing

[1299] Terminal: Resizes a captured photo to a specified resolution (for example, 1024x768 pixels). The input is the captured photo data, which undergoes resolution conversion. Specifically, the application changes the image size internally. The output is the resized image data.

[1300] Device: Sends the resized image to the server. The input is the resized image data, which goes through a process of being sent to the server. Specifically, the device uploads the image to the server using an HTTP request. The output is the image data sent to the server.

[1301] Step 3:

[1302] Image recognition and product identification

[1303] Server: Prepares the received image for analysis. The input is the transmitted image data, which is converted into an internal format so that it can be analyzed. The output is image data that can be analyzed.

[1304] Server: Identifies product categories from images using a convolutional neural network (CNN). The input is analyzable image data, features are extracted using a CNN model, and product categories are identified using a classification model. Specifically, the CNN is executed to identify categories such as "2005 model jackets from a certain brand." The output is the identified product category information.

[1305] Server: Retrieves the market price from the database based on the identified product category. The input is the product category information, and a database query is made based on this. The output is the market price (e.g., 5000 yen).

[1306] Step 4:

[1307] Detecting scratches and stains

[1308] Server: Uses image processing algorithms to detect scratches and stains from product images. The input is analyzable image data, and the image processing algorithm identifies areas of physical damage or stains. Specifically, the algorithm detects stains on the chest of a jacket. The output is information about the detected scratches and stains (area and degree).

[1309] Server: Adjusts the market price based on the detected scratches and stains. The input is the scratch and stain information and the market price, and a comprehensive evaluation is performed to calculate the final price. Specifically, the market price is adjusted to 4000 yen due to the influence of stains. The output is the final price after adjustment.

[1310] Step 5:

[1311] Emotion Analysis

[1312] Emotion engine: Analyzes facial expressions from the user's camera footage in real time to identify the user's emotional state. The input is the user's camera footage, to which an expression analysis algorithm is applied. Specifically, it analyzes the user's facial expressions when viewing the assessment results and determines whether the user is expressing happiness, sadness, surprise, etc. The output is information about the user's emotional state.

[1313] Emotion engine: Analyzes the user's voice to enhance emotional information. The input is the user's voice data, and the voice analysis algorithm supplements the emotional state. The output is integrated emotional state information.

[1314] Step 6:

[1315] Generate offers based on final price and sentiment

[1316] Server: Generates individual proposals based on the adjusted price and the user's emotional information. The input is the final price and the integrated emotional state information. Specifically, the server generates a proposal based on the adjusted price (4000 yen) and the emotional information, such as "We recommend selling this jacket, and the current market price is about 4000 yen. Would you like to sell it?" If the emotion is negative, it also presents an alternative suggestion such as "One option is to consider keeping it." The output is the generated proposal.

[1317] Step 7:

[1318] Displaying the results

[1319] User: Checks the proposal results from the server through the application. The input is the proposal content from the server. The specific operation is that the proposal results are displayed on the smartphone screen. The output is the confirmed proposal results.

[1320] User: Selects the next action based on the suggestion. The input is the suggestion result, and based on that, the user selects an action such as "sell" or "keep." Specific actions are performed by the user tapping an option on the screen. The output is the selected action.

[1321] (Example prompt)

[1322] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[1323] (Application example 2)

[1324] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1325] Conventional food delivery services have been unable to provide personalized menu suggestions or special offers based on the user's emotional state, which makes it difficult for users to receive appropriate service that matches their mood and emotions at the time, potentially resulting in lower satisfaction.

[1326] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for identifying the user's emotion and adjusting the proposal content based on the identification result, means for analyzing product images and identifying the product category, and means for retrieving market prices from a database based on the identified product category. This makes it possible to provide personalized menu suggestions and special offers based on the user's emotional state.

[1327] "Means for users to take or upload product images" refers to devices or software that provide a function for users to take product images and upload them to a server through an application.

[1328] The "means for transmitting the product image to the server" is a communication interface for transmitting the photographed or uploaded product image to the server via the Internet.

[1329] "Means for the server to analyze the product image and identify the product category" refers to a process in which the server uses an image analysis algorithm on the product image received to identify the category to which the product belongs.

[1330] The "means for the server to obtain market price information from the database based on the identified product category" is a process in which the server obtains market price information that matches the identified product category from the database.

[1331] "Means for the server to detect scratches or stains from product images and adjust the market price" refers to a process in which the server analyzes product images, detects the presence or absence of scratches or stains and their extent, and adjusts the market price.

[1332] The "means for the server to propose the adjusted price and disposal method to the user" is a process in which the server proposes to the user, along with the adjusted price information, a disposal method such as whether to sell, keep, or discard the product.

[1333] The "means for the user to select the next action based on the proposal" is an interface that allows the user to receive a proposal from the server and select the next action to be taken (sell, keep, discard, etc.).

[1334] "Means for identifying the user's emotions and adjusting the content of suggestions based on the identification results" refers to a process in which an emotion engine is used to identify emotions from the user's facial expressions and voice, and the content of suggestions is personalized based on the results.

[1335] The present invention relates to a personalized menu recommendation system that utilizes an emotion engine to improve user experience in food delivery services. Specific embodiments for implementing this system will be described below.

[1336] 1. System Configuration

[1337] User terminal

[1338] A device used by a user, such as a smartphone or tablet, that includes a camera and microphone, allowing the user to register emotions through facial expressions and voice, and has an application installed on it that provides an interface for launching the application and placing a delivery order.

[1339] server

[1340] This is a central computer system for analyzing product images, retrieving market prices from a database, detecting scratches and stains, adjusting prices, suggesting disposal methods, and making suggestions based on user emotion recognition. This includes an emotion engine, image analysis algorithms, and database access.

[1341] Database

[1342] This is a system that stores information such as the market price, category, and handling method of a product.

[1343] Emotion Engine

[1344] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[1345] 2. Hardware and software used

[1346] Hardware

[1347] Smartphones and tablets (with cameras and microphones)

[1348] software

[1349] Emotion engine: OpenCV (facial expression analysis), Google Speech-to-Text API (voice analysis)

[1350] Database: MongoDB (Management of menu / emotion pairing data)

[1351] Server: AWS Lambda (serverless environment), Amazon RDS (data storage)

[1352] 3. Data processing and calculation

[1353] User terminal

[1354] Emotional data is collected by users showing facial expressions to the camera and inputting voice data through a microphone, and this data is sent to a server via the application.

[1355] Emotion analysis

[1356] The facial expression data received by the server is analyzed by OpenCV, and the voice data is converted to text using the Google Speech-to-Text API, which allows for specific identification of the user's emotional state.

[1357] Database Access and Proposal Generation

[1358] The emotion engine retrieves appropriate menus from the database according to the user's emotional state, for example, suggesting menus that match a specific emotional state (e.g., if you are tired, suggesting menus that will cheer you up).

[1359] Suggestions for users

[1360] The server sends the menu suggestions along with the adjusted price information to the user's device, which is then displayed on the application to help the user choose the best course of action.

[1361] 4. Specific Examples

[1362] Specific examples of menu suggestions

[1363] 1. The user smiles at the camera.

[1364] 2. Saying "I'm tired today" on the microphone.

[1365] 3. OpenCV analyzes whether or not a smile is present and scores the degree of joy.

[1366] 4. The Google Speech-to-Text API converts "I'm tired today" into text and feeds it back into the emotion engine.

[1367] 5. From the database, a "nutritious chicken salad bowl" is suggested as a menu suitable for the "tired" and "happy" scores.

[1368] 6. The suggested menu is displayed within the app and the user taps the "Order" button.

[1369] Examples of prompt statements

[1370] This prompt is used for the generative AI model:

[1371] Suggest cheer-up foods when the user says "I'm feeling down today" and smiles at the camera. Describe how your system personalizes the suggestions and reassessss whether the user is satisfied with them. Use an emotion engine to analyze the user's emotional feedback in real time and adjust the suggestions accordingly. Use smartphone hardware and simulate the processing flow using OpenCV and the Google Speech-To-Text API.

[1372] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1373] Step 1:

[1374] The user launches the food delivery app. The user shows their facial expression in front of the camera and inputs voice using the microphone. The input data is the user's facial image and voice input. The facial image and voice input are saved on the device and prepared for transmission to the server.

[1375] Step 2:

[1376] The device sends the facial expression images and voice input data it has saved to the server. The data input here are facial expression images and voice files, and communication is performed to send them to the server. This allows the server to begin analyzing the user's emotional data.

[1377] Step 3:

[1378] The server analyzes the received facial image using OpenCV and identifies the emotion from the user's facial expression. The input is the facial image, and the output is an emotion score (e.g., joy, sadness, surprise, etc.). The OpenCV algorithm extracts facial features and scores the emotion based on them.

[1379] Step 4:

[1380] The server converts the received voice data into a string using the Google Speech-to-Text API. The input is voice data and the output is text. Speech recognition technology converts the voice data into text data and provides additional information about the user's emotions.

[1381] Step 5:

[1382] The server combines the results of facial expression analysis and voice analysis to determine the overall emotional state. The input is the emotion score and text data, and the output is the overall emotional state (e.g., tired but happy). The emotion engine combines these data to determine the emotional state with greater accuracy.

[1383] Step 6:

[1384] The server selects an appropriate menu from a database based on the overall emotional state. The input is the emotional state, and the output is the corresponding menu information (e.g., candidates for energizing foods). A database search is performed to obtain menus that match the emotion.

[1385] Step 7:

[1386] The server sends the selected menu information to the terminal. The input here is the menu information, and the output is a menu suggestion that is displayed on the user's terminal. The suggested menu and the reason for it are displayed in the user application.

[1387] Step 8:

[1388] The user reviews and selects a menu suggestion from the server. The input here is the suggested menu, and the output is the user's choice (e.g., confirming the order). The user browses the suggested menu on the app and taps a button to confirm the order.

[1389] Step 9:

[1390] The server stores the user's selections and processes the order. The input is the user's selections and the output is the actual order data. The server stores the order information and issues the order to the food delivery service.

[1391] Step 10:

[1392] After the server places an order, it collects user feedback and adds it to the training data to improve the accuracy of the emotion engine. The input here is user feedback, and the output is an updated emotion data model. User feedback is collected periodically and the emotion engine model is updated.

[1393] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1394] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1395] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1396] [Fourth embodiment]

[1397] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1398] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1399] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1400] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1401] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1402] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1403] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1404] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1405] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1406] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1407] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1408] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1409] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1410] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system consists of a server, a terminal, and a user. The program processing of this system is explained below in natural language. Specific examples of use are also included.

[1411] System Configuration

[1412] 1. User Device

[1413] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[1414] 2. Server

[1415] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[1416] 3. Database

[1417] This is a system that stores information such as the market price, category, and handling method of a product.

[1418] Program processing flow (overview)

[1419] 1. Initial Setup and User Interface Display

[1420] User: Launches the app and takes or uploads a product image.

[1421] Terminal: Accepts images and sends them to the server.

[1422] 2. Image transmission and preprocessing

[1423] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[1424] Server: Receives images and prepares them for analysis.

[1425] 3. Image Recognition and Product Identification

[1426] Server: Identifies product categories using AI techniques (e.g., convolutional neural networks).

[1427] Server: Retrieve the market price for the identified item from the database.

[1428] 4. Detecting scratches and stains

[1429] Server: Uses image processing algorithms to detect scratches and stains from product images.

[1430] Server: Adjusts the market price based on the degree of damage or dirt detected.

[1431] 5. Generate final price and proposal

[1432] Server: Proposes the adjusted price and the optimal disposal method (sell, keep, or destroy) to the user.

[1433] 6. Displaying the results

[1434] User: Check the proposed results.

[1435] User: Choose next action based on suggestions.

[1436] Specific examples

[1437] Specific examples of product appraisal usage

[1438] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[1439] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[1440] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[1441] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[1442] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[1443] 6. Server: Adjust price to 4000 based on the impact of this stain.

[1444] 7. Server: Generate a suggestion: "This jacket is worth selling. The current market price is about 4000 yen. Would you like to sell it?"

[1445] 8. User: Check the proposal results and select "Sell."

[1446] This system allows users to obtain accurate valuation information for unwanted items and choose the best course of action to dispose of them efficiently.

[1447] The processing flow will be explained below.

[1448] Step 1:

[1449] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[1450] Step 2:

[1451] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[1452] Step 3:

[1453] On the device: The resized image data is sent to the server using an HTTP request.

[1454] Step 4:

[1455] Server: Decodes the received image data into an analyzable format.

[1456] Step 5:

[1457] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[1458] Step 6:

[1459] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[1460] Step 7:

[1461] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[1462] Step 8:

[1463] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[1464] Step 9:

[1465] Server: Determines the adjusted price and the proposed disposal method (sell, keep, or destroy), and generates a message to the user. For example, it creates a message saying, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?"

[1466] Step 10:

[1467] User: Check the suggestions displayed on the app screen.

[1468] Step 11:

[1469] User: Selects an action based on the suggestion, for example, "Sell."

[1470] Step 12:

[1471] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[1472] Example 1

[1473] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1474] Currently, systems that appraise products and suggest appropriate disposal methods face challenges in accurately identifying product categories and detecting scratches and stains. In particular, the accuracy of image analysis is low, and appraisal results and disposal method suggestions can be inaccurate, making it difficult for users to obtain reliable information. Another problem is the poor usability of users when entering information into the system.

[1475] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1476] In this invention, the server includes: a means for a user to photograph or upload a product image; a means for transmitting the product image to the server; a means for the server to analyze the product image and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product image and adjust the market price; a means for the server to suggest the adjusted price and a disposal method to the user; a means for the user to select a next action based on the suggestion; a means for the server to use a generative AI model for image analysis; and a means for the user to provide a specific prompt sentence as input. This enables the user to accurately identify the product category and detect scratches or stains, and to receive a highly reliable appraisal result and a suggestion of an appropriate disposal method.

[1477] "User" refers to an individual or corporation that uses the System to take or upload product images and receive appraisals and disposal method suggestions.

[1478] "Product Image" refers to digital image data that a user photographs or uploads to represent an unwanted item.

[1479] "Server" refers to a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and proposes price adjustments and disposal methods.

[1480] "Database" refers to an information management system that stores information such as product categories, market prices, and handling methods.

[1481] A "generative AI model" refers to a trained model that uses artificial intelligence technology to perform image analysis and identify product categories.

[1482] A "prompt sentence" refers to a sentence that allows a user to input specific instructions or information into a system.

[1483] A "convolutional neural network" is a type of deep learning algorithm used in image analysis, and refers to a technology that automatically extracts features from images.

[1484] "Image processing algorithm" refers to the computational procedures or programs used to detect flaws or stains from product images.

[1485] "Quoted Price" refers to the expected price in the market for a particular product category, as retrieved from a database.

[1486] A "proposal message" refers to a message generated by the server to present the adjusted price and the optimal disposal method to the user.

[1487] The present invention is a system that evaluates unwanted items owned by users and proposes optimal disposal methods. This system is composed of a server, terminals, and users. The program processing of this system is explained below.

[1488] The system hardware will use smartphones and tablets as user devices, a central computer system as a server, and an information management system as a database, while the software will include generative AI models and image processing algorithms.

[1489] First, the user uses a smartphone or tablet device to take a photo of the item they want to appraise or upload an existing image. This operation is performed via a dedicated application installed on the device.

[1490] The user device resizes the received image to an appropriate resolution and sends it to the server, which receives the image and uses a generative AI model (e.g., a convolutional neural network) to identify the product category. It then retrieves the market price of the identified product from a database.

[1491] The server then uses image processing algorithms to detect scratches and stains from the product images. Based on this information, the server adjusts the market price. Finally, the server generates a proposal message suggesting the adjusted price and the optimal disposal method (e.g., sell, keep, or destroy) and sends it to the user's device.

[1492] The user can check the suggestion message displayed on the terminal and select the next action based on the suggestion. An example of a specific prompt sentence is, "I uploaded a photo of an old jacket. Please check for defects and stains and let me know the market price."

[1493] Using this system, users can obtain accurate appraisal information for unwanted items and select the optimal action for efficient disposal. For example, if a user wants to have an old jacket appraised, they launch the app, take a photo, and upload it. The system analyzes the image and identifies the jacket's category and market price. It also detects a small stain on the chest, adjusts the price, and suggests selling the item. In this way, the present invention can provide users with highly accurate appraisals and convenient disposal method suggestions.

[1494] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1495] Step 1:

[1496] A user takes or uploads a product image.

[1497] Input: Product images taken by a user using a smartphone or tablet, or existing image data.

[1498] How it works: A user launches the app and either takes a photo of the item they want to appraise or selects and uploads an image from their gallery.

[1499] Output: The product image is displayed on the terminal and the button to proceed to the next step is activated.

[1500] Step 2:

[1501] The device preprocesses the uploaded image and sends it to the server.

[1502] Input: A user-uploaded image.

[1503] Data processing: The device resizes the received image to an appropriate resolution (e.g., 1024x768 pixels).

[1504] How it works: The device resizes the image and compresses it while preserving quality.

[1505] Output: The preprocessed image is sent to the server.

[1506] Step 3:

[1507] The server analyzes the image and identifies the product category.

[1508] Input: The preprocessed image received by the server.

[1509] Data computation: The server performs image analysis to identify product categories using a generative AI model (e.g., convolutional neural network).

[1510] How it works: A convolutional neural network extracts features from an image and matches them with information in a database.

[1511] Output: Identified product category (e.g., "Brand name jacket (2005 model)").

[1512] Step 4:

[1513] The server retrieves market prices from a database based on the identified product category.

[1514] Input: Identified product category.

[1515] Data acquisition: The server searches the database for the market price corresponding to the product category and acquires it.

[1516] How it works: The server generates a database query to retrieve detailed product information, including price quotes.

[1517] Output: Market price of the item (e.g., 5000 yen).

[1518] Step 5:

[1519] The server detects scratches and stains from product images and adjusts the market price.

[1520] Input: Product image and quote price.

[1521] Data processing: The server uses image processing algorithms to detect scratches and stains in the product images and adjust the market price.

[1522] How it works: Image analysis algorithms detect defects in images and recalculate prices based on that information.

[1523] Output: The adjusted price (e.g., $4000).

[1524] Step 6:

[1525] The server proposes the adjusted price and disposal method to the user.

[1526] Input: Adjusted price.

[1527] Data Generation: The server generates a proposal message containing the adjusted price and the optimal disposal method.

[1528] Behavior: The server creates a suggestion message that reads, "This jacket is recommended for sale. The current market price is about $40.00. Would you like to sell it?"

[1529] Output: The proposal message is sent to the terminal.

[1530] Step 7:

[1531] The user selects the next action based on the suggestions.

[1532] Input: The suggestion message displayed on the terminal.

[1533] Action: The user reviews the proposal and selects "Sell."

[1534] Output: The user's selection is communicated to the server, and the next action is performed.

[1535] (Application example 1)

[1536] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1537] Conventional systems that assess unwanted products and suggest disposal methods do not support frequently used ingredients or food delivery services. Furthermore, there are no tools that can quickly assess the freshness and condition of ingredients in the refrigerator and suggest optimal ways to use or dispose of them, leaving users without a way to easily reduce food waste. This results in a large amount of food waste within the home, which increases costs and puts a strain on the environment.

[1538] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1539] In this invention, the server includes: a means for a user to photograph or upload product images; a means for transmitting the product images to the server; a means for the server to analyze the product images and identify a product category; a means for the server to obtain a market price from a database based on the identified product category; a means for the server to detect scratches or stains from the product images and adjust the market price; a means for a user to photograph or upload an image of an ingredient; a means for transmitting the ingredient image to the server; a means for the server to analyze the ingredient image and identify a food category; a means for the server to obtain an expiration date and an optimal usage method from a database based on the identified food category; and a means for the server to detect the freshness and condition of the ingredient from the ingredient image and adjust the usage method. This allows a user to easily check the freshness of ingredients in the refrigerator and quickly find optimal usage and disposal methods to reduce waste.

[1540] A "user terminal" is a device that a user uses to take or upload images of products or ingredients.

[1541] The "server" is a central computer system that analyzes images of products and ingredients, identifies categories, adjusts prices, and detects freshness and condition.

[1542] A "database" is a system that stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[1543] "Product category" refers to the type or classification of a product identified by the server through analysis.

[1544] "Quoted Price" is the market price of the commodity obtained from the database.

[1545] "Scratches and stains" refer to external defects and stains detected from product images.

[1546] "Food image" is an image taken or uploaded by the user to understand the condition of the food in the refrigerator.

[1547] "Freshness" refers to the state of freshness of ingredients and storage conditions.

[1548] "Use by date" is the expiration date of the ingredient obtained from the database.

[1549] The "optimal use" is a method of use suggested by the server based on the freshness and condition of the ingredients.

[1550] The "adjusted price" is the adjusted market price calculated by the server based on the results of detecting scratches and stains.

[1551] The "suggestion result" is information about the adjusted price and how to use the ingredients that is displayed to the user.

[1552] An "action" is the next operation that the user selects based on the suggested results.

[1553] This invention is a system that analyzes the status of unwanted products and food items stored in a user's refrigerator and suggests optimal disposal and usage methods. The system is composed of a server, a user terminal, and a database.

[1554] System Configuration

[1555] User terminal

[1556] It is a device such as a smartphone or tablet that provides an interface for taking and uploading images of products and ingredients and operating applications.

[1557] server

[1558] It is a central computer system that analyzes images of products and ingredients, identifies categories, obtains market prices, detects scratches, dirt, freshness and condition, adjusts prices, and suggests ways to use and dispose of them. Specifically, it makes full use of convolutional neural networks (CNNs) using TensorFlow and Keras.

[1559] Database

[1560] This system stores information such as the market price, category, expiration date, and handling method of products and ingredients.

[1561] Program processing flow

[1562] User terminal

[1563] The user takes or uploads an image of the product or ingredient.

[1564] The image is resized to the appropriate resolution and sent to the server.

[1565] server

[1566] The server receives the image and analyzes it using a convolutional neural network (CNN).

[1567] For products, the system identifies the category, obtains the market price, detects scratches and stains, and proposes an adjusted price and disposal method.

[1568] In the case of food ingredients, the system identifies the food category, obtains the expiration date and optimal usage method, detects the freshness and condition, and suggests how to use and dispose of the food.

[1569] Specific examples

[1570] Specific examples of product appraisal usage

[1571] 1. The user takes a photo of an old jacket (e.g., a 2005 model from Brand A), launches the app, and uploads the image.

[1572] 2. The device resizes the image and sends it to the server.

[1573] 3. The server analyzes the image, determines the market price of the jacket, detects scratches and stains, and calculates the adjusted price.

[1574] 4. The server generates a suggestion such as "We recommend selling this jacket. The market price is about 4,000 yen. Would you like to sell it?" and displays the suggestion result to the user.

[1575] 5. The user selects "Sell" based on the proposed results.

[1576] Food ingredient management example

[1577] 1. The user takes a photo of an apple in the refrigerator, launches the app and uploads the image.

[1578] 2. The device resizes the image and sends it to the server.

[1579] 3. The server analyzes the image, identifies the apple category, determines its freshness, and retrieves expiration date information from a database.

[1580] 4. The server generates a suggestion such as "This apple is fresh. We recommend eating it raw." and displays the suggestion result to the user.

[1581] 5. The user selects the next action to take based on the suggested results.

[1582] Prompt Sentence Examples

[1583] "Just take a photo of the food in your refrigerator and upload it to the app. The system will automatically determine the freshness of the food and tell you the best way to use it. For example, if you upload a photo of an apple, it will suggest, 'This apple is fresh. We recommend eating it raw.'"

[1584] As described above, the present invention provides a system that allows users to efficiently manage goods and ingredients by analyzing images of the goods and ingredients and proposing optimal disposal and usage methods.

[1585] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1586] Step 1:

[1587] The user takes or uploads an image of the product or ingredient.

[1588] Input: An image of a product or ingredient.

[1589] Specific operation: The user uses a user device such as a smartphone or tablet to take a photo of the target product or ingredient or upload an existing image.

[1590] Output: Image data captured or uploaded.

[1591] Step 2:

[1592] The terminal resizes the product or ingredient image to an appropriate resolution and transmits it to the server.

[1593] Input: Image data captured or uploaded.

[1594] Specific operation: The device resizes the image resolution to, for example, 1024x768 pixels and sends the image data to the server.

[1595] Output: The resized image data.

[1596] Step 3:

[1597] The server receives the images and analyzes them using a convolutional neural network (CNN).

[1598] Input: The resized image data.

[1599] Specific operation: An AI model (using TensorFlow and Keras) installed on the server reads the image and analyzes it using CNN to identify the product or ingredient category.

[1600] Output: Category information of products and ingredients.

[1601] Step 4:

[1602] The server retrieves the market price and expiration date from the database based on the identified category.

[1603] Input: Category information of product or ingredients.

[1604] Specific operation: The server accesses the database and retrieves the market price based on the identified product category, the expiration date based on the food ingredient category, and the latest information.

[1605] Output: Market price information for products or expiration date information for ingredients.

[1606] Step 5:

[1607] The server detects scratches and dirt from product images and detects freshness and condition from food ingredient images.

[1608] Input: Resized image data and category information.

[1609] How it works: The server uses image processing algorithms to detect scratches and dirt from product images, and freshness and condition from food images.

[1610] Output: Information on detected scratches and stains, freshness and condition.

[1611] Step 6:

[1612] Based on the detection results, the server will suggest product price adjustments and optimal ways to use and dispose of ingredients.

[1613] Input: Market price information or expiration date information, damage or stain information, freshness or condition information.

[1614] What it does: The server adjusts the market price based on the results of any damage or stains detected, and determines the best way to use or dispose of the ingredients.

[1615] Output: Adjusted price or recommendations on optimal usage and disposal methods.

[1616] Step 7:

[1617] The user selects the next action based on the suggestions.

[1618] Input: Proposal information.

[1619] Specific operation: The user checks the proposal results on their device and selects the next action (e.g., sell the product, use the ingredients for a delicious meal, or discard if unnecessary).

[1620] Output: The result of the selection of the next action to be taken.

[1621] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1622] This invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained in natural language below. Specific examples of use are also included.

[1623] System Configuration

[1624] 1. User Device

[1625] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[1626] 2. Server

[1627] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[1628] 3. Database

[1629] This is a system that stores information such as the market price, category, and handling method of a product.

[1630] 4. Emotion Engine

[1631] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[1632] Program processing flow (overview)

[1633] 1. Initial Setup and User Interface Display

[1634] User: Launch the app and take or upload a photo of the item you want appraised.

[1635] Terminal: Receives the image and sends it to the server.

[1636] 2. Image transmission and preprocessing

[1637] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[1638] Server: Receives images and prepares them for analysis.

[1639] 3. Image Recognition and Product Identification

[1640] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[1641] Server: Retrieve the market price for the identified item from the database.

[1642] 4. Detecting scratches and stains

[1643] Server: Uses image processing algorithms to detect scratches and stains from product images.

[1644] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[1645] 5. Emotion Analysis

[1646] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[1647] 6. Proposal generation based on final price and sentiment

[1648] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[1649] 7. Displaying the results

[1650] User: Check the proposed results.

[1651] User: Choose next action based on suggestions.

[1652] Specific examples

[1653] Specific use cases for product appraisal and emotion recognition

[1654] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[1655] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[1656] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[1657] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[1658] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[1659] 6. Server: Adjust price to 4000 based on the impact of this stain.

[1660] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[1661] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[1662] 9. User: Check the proposal results and select "Sell" or "Keep."

[1663] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[1664] The processing flow will be explained below.

[1665] Step 1:

[1666] User: Launches the app and takes a photo of the item they want appraised, or selects an existing photo from their gallery.

[1667] Step 2:

[1668] Device: Resize the captured or selected image to the appropriate resolution (e.g., 1024x768 pixels) within the app.

[1669] Step 3:

[1670] On the device: The resized image data is sent to the server using an HTTP request.

[1671] Step 4:

[1672] Server: Decodes the received image data into an analyzable format.

[1673] Step 5:

[1674] Server: Uses a convolutional neural network (CNN) to identify the product category (clothing, miscellaneous goods, electronics, furniture) from the image. For example, it identifies the product as a "jacket from a certain brand."

[1675] Step 6:

[1676] Server: Based on the identified product category, retrieve the market price from the database. For example, estimate the market price of a certain brand jacket (2005 model) to be 5,000 yen.

[1677] Step 7:

[1678] Server: Image processing algorithms are used to detect scratches and stains from product images. Here, edge detection and segmentation techniques are used to check for small blemishes on the surface of the product.

[1679] Step 8:

[1680] Server: Evaluate the extent and severity of the detected scratches and stains and take them into account as factors that affect the market price. For example, a stain on the chest could adjust the market price from ¥5,000 to ¥4,000.

[1681] Step 9:

[1682] Emotion Engine: Analyzes facial expressions in real time from the user's camera to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[1683] Step 10:

[1684] Server: Based on the user's emotional information obtained from the emotion engine, the server proposes an adjusted price and the optimal disposal method (sell, keep, or destroy). For example, if the user shows signs of being emotionally reluctant to sell, the server will suggest keeping the item.

[1685] Step 11:

[1686] Server: Generates a suggestion message to the user, such as "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" or "Or, consider keeping it," and displays it on the screen.

[1687] Step 12:

[1688] User: Check the proposed results.

[1689] Step 13:

[1690] User: Selects an action based on the suggestion, for example, "sell" or "keep."

[1691] Step 14:

[1692] Terminal: Notifies the server of the user's selection and performs the next process (e.g., starts the sale procedure).

[1693] Example 2

[1694] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1695] Conventional unwanted item appraisal systems have the problem that it is difficult to accurately evaluate the condition of products and do not provide personalized suggestions that take the user's emotions into consideration, resulting in low user satisfaction. This invention aims to achieve more accurate appraisals and suggestions that result in higher user satisfaction by combining product image analysis and an emotion engine.

[1696] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing a product image and identifying a product category, a means for acquiring a market price from a database based on the identified product category, a means for detecting scratches or stains from the product image and adjusting the market price, a means for analyzing the user's emotional state using an emotion engine, and a means for adjusting the proposal content based on the user's emotional state. This enables more accurate appraisals and personalized proposals that take the user's emotions into consideration.

[1697] "Product image" refers to a photograph or image data of the product for which the user wishes to have it appraised.

[1698] A "server" is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, and generates suggestions in cooperation with an emotion engine.

[1699] "Product category" is information that indicates the type or classification of a specific product, and by identifying it, it becomes possible to obtain the market price.

[1700] "Quoted Price" refers to the prevailing price in the market for a particular commodity, and is obtained from a database.

[1701] A "database" refers to an information system that stores information such as the market price of a product, category information, and handling instructions.

[1702] "Scratches and stains" refers to physical damage or stains on the surface of the product, which are factors that affect the value of the product.

[1703] An "emotion engine" refers to an algorithm or system that analyzes a user's facial expressions and voice to recognize their emotional state.

[1704] "Adjusting the proposal" refers to optimizing the proposal for how to dispose of the product and the final price based on the user's emotional state.

[1705] An "action" refers to a decision or action (e.g., sell, keep, or discard) that a user makes based on a suggestion from the server.

[1706] The present invention combines an emotion engine with a system that assesses unwanted items and suggests optimal disposal methods. This system consists of a user, a terminal, a server, and an emotion engine. The program processing of this system is explained below in natural language.

[1707] System Configuration

[1708] 1. User Device

[1709] A device used by a user, such as a smartphone or tablet, that provides an interface for taking photos and operating applications.

[1710] 2. Server

[1711] It is a central computer system that analyzes product images, retrieves market prices from a database, detects scratches and stains, adjusts prices, suggests disposal methods, and makes suggestions based on user emotion recognition.

[1712] 3. Database

[1713] This is a system that stores information such as the market price, category, and handling method of a product.

[1714] 4. Emotion Engine

[1715] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[1716] Program Processing Overview

[1717] The program operates as follows.

[1718] 1. Initial Setup and User Interface Display

[1719] User: Launch the app and take or upload a photo of the item you want appraised.

[1720] Terminal: Receives the image and sends it to the server.

[1721] 2. Image transmission and preprocessing

[1722] On the device: Uploaded images are converted to the appropriate resolution and sent to the server.

[1723] Server: Receives images and prepares them for analysis.

[1724] 3. Image Recognition and Product Identification

[1725] Server: Uses a convolutional neural network (CNN) to identify product categories from images.

[1726] Server: Retrieve the market price for the identified item from the database.

[1727] 4. Detecting scratches and stains

[1728] Server: Uses image processing algorithms to detect scratches and stains from product images.

[1729] Server: Evaluates the extent and severity of any scratches or stains detected and takes them into account as factors that affect the market price.

[1730] 5. Emotion Analysis

[1731] Emotion Engine: Analyzes facial expressions from the user's camera in real time to identify the user's emotional state (happiness, sadness, surprise, etc.), and further augments emotional information through voice analysis.

[1732] 6. Proposal generation based on final price and sentiment

[1733] Server: Based on the adjusted price and the user's emotional information obtained from the emotion engine, the server proposes a more personalized disposal method (sell, keep, or destroy). For example, if the user shows signs of emotional reluctance to sell, the server will suggest keeping the item.

[1734] 7. Displaying the results

[1735] User: Check the proposed results.

[1736] User: Choose next action based on suggestions.

[1737] Specific examples

[1738] Specific use cases for product appraisal and emotion recognition

[1739] 1. User: I have an old jacket (a 2005 model from a certain brand). I launch the app, take a photo of the jacket, and upload it.

[1740] 2. Terminal: Resize the uploaded image to 1024x768 pixels and send it to the server.

[1741] 3. Server: Receives the image and uses a convolutional neural network to identify the jacket as a "certain brand jacket (2005 model)."

[1742] 4. Server: Gets the going price for this jacket from the database and estimates it to be $5000.

[1743] 5. Server: Image processing algorithms detect stains on the jacket, for example identifying a small stain on the chest.

[1744] 6. Server: Adjust price to 4000 based on the impact of this stain.

[1745] 7. Emotion Engine: Analyzes the user's facial expressions when viewing the assessment results and evaluates whether the user is satisfied. For example, if the user shows dissatisfaction or sadness about the proposal, the emotion information is sent to the server.

[1746] 8. Server: Based on the adjusted price and the user's sentiment, the server generates a suggestion such as, "This jacket is recommended for sale, and the current market price is about 4,000 yen. Would you like to sell it?" If the user's sentiment is negative about selling it, the server also presents an alternative suggestion such as, "One option is to consider keeping it."

[1747] 9. User: Check the proposal results and select "Sell" or "Keep."

[1748] This system not only provides users with accurate valuation information for unwanted items, but also provides personalized suggestions that take their emotional state into account, allowing them to make more satisfying decisions.

[1749] Specific prompt examples

[1750] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[1751] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1752] Step 1:

[1753] Initial Setup and User Interface Display

[1754] User: Launches the app and takes a photo of the item they wish to have appraised. The input is a photo of the item. Specifically, the user taps the smartphone screen to launch the app, selects camera mode, and takes a photo. The output is the captured photo data.

[1755] Step 2:

[1756] Image transmission and preprocessing

[1757] Terminal: Resizes a captured photo to a specified resolution (for example, 1024x768 pixels). The input is the captured photo data, which undergoes resolution conversion. Specifically, the application changes the image size internally. The output is the resized image data.

[1758] Device: Sends the resized image to the server. The input is the resized image data, which goes through a process of being sent to the server. Specifically, the device uploads the image to the server using an HTTP request. The output is the image data sent to the server.

[1759] Step 3:

[1760] Image recognition and product identification

[1761] Server: Prepares the received image for analysis. The input is the transmitted image data, which is converted into an internal format so that it can be analyzed. The output is image data that can be analyzed.

[1762] Server: Identifies product categories from images using a convolutional neural network (CNN). The input is analyzable image data, features are extracted using a CNN model, and product categories are identified using a classification model. Specifically, the CNN is executed to identify categories such as "2005 model jackets from a certain brand." The output is the identified product category information.

[1763] Server: Retrieves the market price from the database based on the identified product category. The input is the product category information, and a database query is made based on this. The output is the market price (e.g., 5000 yen).

[1764] Step 4:

[1765] Detecting scratches and stains

[1766] Server: Uses image processing algorithms to detect scratches and stains from product images. The input is analyzable image data, and the image processing algorithm identifies areas of physical damage or stains. Specifically, the algorithm detects stains on the chest of a jacket. The output is information about the detected scratches and stains (area and degree).

[1767] Server: Adjusts the market price based on the detected scratches and stains. The input is the scratch and stain information and the market price, and a comprehensive evaluation is performed to calculate the final price. Specifically, the market price is adjusted to 4000 yen due to the influence of stains. The output is the final price after adjustment.

[1768] Step 5:

[1769] Emotion Analysis

[1770] Emotion engine: Analyzes facial expressions from the user's camera footage in real time to identify the user's emotional state. The input is the user's camera footage, to which an expression analysis algorithm is applied. Specifically, it analyzes the user's facial expressions when viewing the assessment results and determines whether the user is expressing happiness, sadness, surprise, etc. The output is information about the user's emotional state.

[1771] Emotion engine: Analyzes the user's voice to enhance emotional information. The input is the user's voice data, and the voice analysis algorithm supplements the emotional state. The output is integrated emotional state information.

[1772] Step 6:

[1773] Generate offers based on final price and sentiment

[1774] Server: Generates individual proposals based on the adjusted price and the user's emotional information. The input is the final price and the integrated emotional state information. Specifically, the server generates a proposal based on the adjusted price (4000 yen) and the emotional information, such as "We recommend selling this jacket, and the current market price is about 4000 yen. Would you like to sell it?" If the emotion is negative, it also presents an alternative suggestion such as "One option is to consider keeping it." The output is the generated proposal.

[1775] Step 7:

[1776] Displaying the results

[1777] User: Checks the proposal results from the server through the application. The input is the proposal content from the server. The specific operation is that the proposal results are displayed on the smartphone screen. The output is the confirmed proposal results.

[1778] User: Selects the next action based on the suggestion. The input is the suggestion result, and based on that, the user selects an action such as "sell" or "keep." Specific actions are performed by the user tapping an option on the screen. The output is the selected action.

[1779] (Example prompt)

[1780] "You launch the app, take a photo of the item, upload it, and then choose whether to sell, keep, or destroy it based on the appraisal results and our recommendations."

[1781] (Application example 2)

[1782] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1783] Conventional food delivery services have been unable to provide personalized menu suggestions or special offers based on the user's emotional state, which makes it difficult for users to receive appropriate service that matches their mood and emotions at the time, potentially resulting in lower satisfaction.

[1784] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for identifying the user's emotion and adjusting the proposal content based on the identification result, means for analyzing product images and identifying the product category, and means for retrieving market prices from a database based on the identified product category. This makes it possible to provide personalized menu suggestions and special offers based on the user's emotional state.

[1785] "Means for users to take or upload product images" refers to devices or software that provide a function for users to take product images and upload them to a server through an application.

[1786] The "means for transmitting the product image to the server" is a communication interface for transmitting the photographed or uploaded product image to the server via the Internet.

[1787] "Means for the server to analyze the product image and identify the product category" refers to a process in which the server uses an image analysis algorithm on the product image received to identify the category to which the product belongs.

[1788] The "means for the server to obtain market price information from the database based on the identified product category" is a process in which the server obtains market price information that matches the identified product category from the database.

[1789] "Means for the server to detect scratches or stains from product images and adjust the market price" refers to a process in which the server analyzes product images, detects the presence or absence of scratches or stains and their extent, and adjusts the market price.

[1790] The "means for the server to propose the adjusted price and disposal method to the user" is a process in which the server proposes to the user, along with the adjusted price information, a disposal method such as whether to sell, keep, or discard the product.

[1791] The "means for the user to select the next action based on the proposal" is an interface that allows the user to receive a proposal from the server and select the next action to be taken (sell, keep, discard, etc.).

[1792] "Means for identifying the user's emotions and adjusting the content of suggestions based on the identification results" refers to a process in which an emotion engine is used to identify emotions from the user's facial expressions and voice, and the content of suggestions is personalized based on the results.

[1793] The present invention relates to a personalized menu recommendation system that utilizes an emotion engine to improve user experience in food delivery services. Specific embodiments for implementing this system will be described below.

[1794] 1. System Configuration

[1795] User terminal

[1796] A device used by a user, such as a smartphone or tablet, that includes a camera and microphone, allowing the user to register emotions through facial expressions and voice, and has an application installed on it that provides an interface for launching the application and placing a delivery order.

[1797] server

[1798] This is a central computer system for analyzing product images, retrieving market prices from a database, detecting scratches and stains, adjusting prices, suggesting disposal methods, and making suggestions based on user emotion recognition. This includes an emotion engine, image analysis algorithms, and database access.

[1799] Database

[1800] This is a system that stores information such as the market price, category, and handling method of a product.

[1801] Emotion Engine

[1802] This component analyzes the user's emotions from their facial expressions and voice, and adjusts the suggestions based on that information.

[1803] 2. Hardware and software used

[1804] Hardware

[1805] Smartphones and tablets (with cameras and microphones)

[1806] software

[1807] Emotion engine: OpenCV (facial expression analysis), Google Speech-to-Text API (voice analysis)

[1808] Database: MongoDB (Management of menu / emotion pairing data)

[1809] Server: AWS Lambda (serverless environment), Amazon RDS (data storage)

[1810] 3. Data processing and calculation

[1811] User terminal

[1812] Emotional data is collected by users showing facial expressions to the camera and inputting voice data through a microphone, and this data is sent to a server via the application.

[1813] Emotion analysis

[1814] The facial expression data received by the server is analyzed by OpenCV, and the voice data is converted to text using the Google Speech-to-Text API, which allows for specific identification of the user's emotional state.

[1815] Database Access and Proposal Generation

[1816] The emotion engine retrieves appropriate menus from the database according to the user's emotional state, for example, suggesting menus that match a specific emotional state (e.g., if you are tired, suggesting menus that will cheer you up).

[1817] Suggestions for users

[1818] The server sends the menu suggestions along with the adjusted price information to the user's device, which is then displayed on the application to help the user choose the best course of action.

[1819] 4. Specific Examples

[1820] Specific examples of menu suggestions

[1821] 1. The user smiles at the camera.

[1822] 2. Saying "I'm tired today" on the microphone.

[1823] 3. OpenCV analyzes whether or not a smile is present and scores the degree of joy.

[1824] 4. The Google Speech-to-Text API converts "I'm tired today" into text and feeds it back into the emotion engine.

[1825] 5. From the database, a "nutritious chicken salad bowl" is suggested as a menu suitable for the "tired" and "happy" scores.

[1826] 6. The suggested menu is displayed within the app and the user taps the "Order" button.

[1827] Examples of prompt statements

[1828] This prompt is used for the generative AI model:

[1829] Suggest cheer-up foods when the user says "I'm feeling down today" and smiles at the camera. Describe how your system personalizes the suggestions and reassessss whether the user is satisfied with them. Use an emotion engine to analyze the user's emotional feedback in real time and adjust the suggestions accordingly. Use smartphone hardware and simulate the processing flow using OpenCV and the Google Speech-To-Text API.

[1830] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1831] Step 1:

[1832] The user launches the food delivery app. The user shows their facial expression in front of the camera and inputs voice using the microphone. The input data is the user's facial image and voice input. The facial image and voice input are saved on the device and prepared for transmission to the server.

[1833] Step 2:

[1834] The device sends the facial expression images and voice input data it has saved to the server. The data input here are facial expression images and voice files, and communication is performed to send them to the server. This allows the server to begin analyzing the user's emotional data.

[1835] Step 3:

[1836] The server analyzes the received facial image using OpenCV and identifies the emotion from the user's facial expression. The input is the facial image, and the output is an emotion score (e.g., joy, sadness, surprise, etc.). The OpenCV algorithm extracts facial features and scores the emotion based on them.

[1837] Step 4:

[1838] The server converts the received voice data into a string using the Google Speech-to-Text API. The input is voice data and the output is text. Speech recognition technology converts the voice data into text data and provides additional information about the user's emotions.

[1839] Step 5:

[1840] The server combines the results of facial expression analysis and voice analysis to determine the overall emotional state. The input is the emotion score and text data, and the output is the overall emotional state (e.g., tired but happy). The emotion engine combines these data to determine the emotional state with greater accuracy.

[1841] Step 6:

[1842] The server selects an appropriate menu from a database based on the overall emotional state. The input is the emotional state, and the output is the corresponding menu information (e.g., candidates for energizing foods). A database search is performed to obtain menus that match the emotion.

[1843] Step 7:

[1844] The server sends the selected menu information to the terminal. The input here is the menu information, and the output is a menu suggestion that is displayed on the user's terminal. The suggested menu and the reason for it are displayed in the user application.

[1845] Step 8:

[1846] The user reviews and selects a menu suggestion from the server. The input here is the suggested menu, and the output is the user's choice (e.g., confirming the order). The user browses the suggested menu on the app and taps a button to confirm the order.

[1847] Step 9:

[1848] The server stores the user's selections and processes the order. The input is the user's selections and the output is the actual order data. The server stores the order information and issues the order to the food delivery service.

[1849] Step 10:

[1850] After the server places an order, it collects user feedback and adds it to the training data to improve the accuracy of the emotion engine. The input here is user feedback, and the output is an updated emotion data model. User feedback is collected periodically and the emotion engine model is updated.

[1851] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1852] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1853] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1854] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1855] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1856] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1857] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1858] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1859] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1860] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1861] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1862] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1863] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1864] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1865] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1866] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1867] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1868] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1869] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1870] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1871] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1872] The following is further disclosed regarding the above embodiment.

[1873] (Claim 1)

[1874] A means for users to take or upload product images;

[1875] means for transmitting the product image to a server;

[1876] A means for the server to analyze the product image and identify the product category;

[1877] A means for the server to obtain market prices from a database based on the identified product category;

[1878] A means for a server to detect scratches or stains from product images and adjust the market price;

[1879] A means for the server to propose the adjusted price and disposal method to the user;

[1880] a means for the user to select a next action based on the suggestions;

[1881] A system including:

[1882] (Claim 2)

[1883] 2. The system of claim 1, wherein the detection of scratches and stains is performed using an image processing algorithm.

[1884] (Claim 3)

[1885] 10. The system of claim 1, wherein the server identifies the product category using a convolutional neural network.

[1886] "Example 1"

[1887] (Claim 1)

[1888] A means for users to take or upload product images;

[1889] means for transmitting the product image to a server;

[1890] A means for the server to analyze the product image and identify the product category;

[1891] A means for the server to obtain market prices from a database based on the identified product category;

[1892] A means for a server to detect scratches or stains from product images and adjust the market price;

[1893] A means for the server to propose the adjusted price and disposal method to the user;

[1894] a means for the user to select a next action based on the suggestions;

[1895] a means by which the server uses the generative AI model for image analysis;

[1896] a means for the user to provide a particular prompt sentence as input;

[1897] A system including:

[1898] (Claim 2)

[1899] 2. The system of claim 1, wherein the detection of scratches and stains is performed using an image processing algorithm.

[1900] (Claim 3)

[1901] 10. The system of claim 1, wherein the server identifies the product category using a convolutional neural network.

[1902] "Application Example 1"

[1903] (Claim 1)

[1904] A means for users to take or upload product images;

[1905] means for transmitting the product image to a server;

[1906] A means for the server to analyze the product image and identify the product category;

[1907] A means for the server to obtain market prices from a database based on the identified product category;

[1908] A means for a server to detect scratches or stains from product images and adjust the market price;

[1909] A means for the server to propose the adjusted price and disposal method to the user;

[1910] a means for the user to select a next action based on the suggestions;

[1911] A means for a user to take or upload an image of an ingredient;

[1912] means for transmitting the ingredient image to a server;

[1913] A means for the server to analyze the ingredient image and identify the ingredient category;

[1914] A means for the server to obtain expiration dates and optimal usage methods from a database based on the identified food category;

[1915] A server detects the freshness and condition of the food material from the image and adjusts the method of use;

[1916] A system including:

[1917] (Claim 2)

[1918] 10. The system of claim 1, wherein image processing algorithms are used to detect freshness and condition.

[1919] (Claim 3)

[1920] 2. The system of claim 1, wherein the server identifies the ingredient category using a convolutional neural network.

[1921] "Example 2: Combining Emotion Engines"

[1922] (Claim 1)

[1923] A means for users to take or upload product images;

[1924] means for transmitting the product image to a server;

[1925] A means for the server to analyze the product image and identify the product category;

[1926] A means for the server to obtain market prices from a database based on the identified product category;

[1927] A means for a server to detect scratches or stains from product images and adjust the market price;

[1928] A means for the server to propose the adjusted price and disposal method to the user;

[1929] a means for the user to select a next action based on the suggestions;

[1930] means for analyzing a user's emotional state using an emotion engine;

[1931] a means for the server to adjust the content of the suggestions based on the emotional state of the user;

[1932] A system including:

[1933] (Claim 2)

[1934] 2. The system of claim 1, wherein the detection of scratches and stains is performed using an image processing algorithm.

[1935] (Claim 3)

[1936] 10. The system of claim 1, wherein the server identifies the product category using a convolutional neural network.

[1937] "Application example 2 when combining emotion engines"

[1938] (Claim 1)

[1939] A means for users to take or upload product images;

[1940] means for transmitting the product image to a server;

[1941] A means for the server to analyze the product image and identify the product category;

[1942] A means for the server to obtain market prices from a database based on the identified product category;

[1943] A means for a server to detect scratches or stains from product images and adjust the market price;

[1944] A means for the server to propose the adjusted price and disposal method to the user;

[1945] a means for the user to select a next action based on the suggestions;

[1946] means for identifying a user's emotion and adjusting the content of the suggestions based on the identification result;

[1947] A system including:

[1948] (Claim 2)

[1949] 2. The system of claim 1, wherein the detection of scratches and stains is performed using an image processing algorithm.

[1950] (Claim 3)

[1951] 10. The system of claim 1, wherein the server identifies the product category using a convolutional neural network. [Explanation of symbols]

[1952] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to take or upload product images; means for transmitting the product image to a server; A means for the server to analyze the product image and identify the product category; A means for the server to obtain market prices from a database based on the identified product category; A means for a server to detect scratches or stains from product images and adjust the market price; A means for the server to propose the adjusted price and disposal method to the user; a means for the user to select a next action based on the suggestions; A system including:

2. 10. The system of claim 1, wherein the detection of scratches and stains is performed using an image processing algorithm.

3. 10. The system of claim 1, wherein the server identifies the product category using a convolutional neural network.

Images