System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring a product image; means for analyzing the acquired image and specifying product information; means for automatically generating offer information based on the specified product information; means for causing a user to input insufficient offer information; and means for offering the offer information on a predetermined platform.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When listing items on platforms such as auctions and flea markets, the process of entering product information is cumbersome and many users find it time-consuming. In particular, accurately identifying the product and entering the appropriate price and description takes a significant amount of time, and the product often ends up selling for next to nothing on the market. This situation does not meet the needs of users who want to list items easily, and the liquidity of products is declining, which is an issue. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including a means for acquiring product images, a means for analyzing the acquired images and identifying product information, a means for automatically generating listing information based on the identified product information, a means for prompting a user to input missing listing information, and a means for listing the listing information on a specified platform.

[0006] Specifically, it uses image recognition technology to identify products from photos and uses an existing product database to obtain detailed product information. Based on this, it automatically generates listing titles and descriptions, and provides an interface for users to enter supplementary information about the product's condition and accessories. Furthermore, by using the listing platform's API to automate the listing process, it significantly reduces the burden on users and simplifies the listing process. This also makes it possible to increase product liquidity.

[0007] "Product images" refer to photographic data of the products users wish to sell taken with a smartphone or camera.

[0008] "Image recognition technology" refers to technology that uses computer vision and OCR technology to identify characters and shapes from images and extract specific product information.

[0009] "Product Information" refers to all information related to a product, including the product name, model number, specifications, JAN code, ISBN code, and other identifying information.

[0010] "Listing information" refers to information required when listing an item on an auction or flea market platform, including the item name, description, price, and item condition.

[0011] A "product database" is a database that stores detailed information related to existing products, including product names, model numbers, specifications, images, JAN codes, ISBN codes, etc.

[0012] "Automatic generation" refers to a process in which the system automatically generates information using predefined algorithms, without requiring manual input from the user.

[0013] "User interface" refers to the screen or operating means through which the system and the user exchange information, and in particular to the interface that prompts the user to confirm or enter missing listing information.

[0014] "Listing platform" refers to an online platform where goods are bought and sold, such as at auctions and flea markets, and refers to a system that provides a place for users to buy and sell goods.

[0015] "API" stands for Application Programming Interface, and refers to an interface that allows different software programs to communicate with each other. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that significantly reduces the effort required to put up an item at an auction or flea market. A user photographs an item and uploads the image to the system, which then automatically identifies the item information and generates listing information. Specific embodiments of this system are described below.

[0038] Overall system overview

[0039] 1. Product image acquisition

[0040] Users take a photo of the product they want to sell using a smartphone or digital camera.

[0041] The product images taken by the terminal are uploaded to the system.

[0042] 2. Image Recognition and Product Identification

[0043] The server receives the uploaded image and identifies the product using image recognition technology.

[0044] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[0045] 3. Obtaining product information

[0046] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[0047] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[0048] 4. Automatic generation of listing information

[0049] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[0050] The terminal displays the automatically generated listing information to the user.

[0051] 5. Information Complementary Interface

[0052] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[0053] The user enters the requested information and provides final confirmation.

[0054] 6. Completing the listing process

[0055] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[0056] The server uses the listing platform's API to complete the listing process.

[0057] Specific examples

[0058] For example, if a user wants to sell a Canon EOS 80D camera:

[0059] 1. The user takes a photo with the camera on their smartphone.

[0060] 2. Upload the images taken by the device to the system server.

[0061] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[0062] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[0063] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[0064] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[0065] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[0066] 8. The server processes the listing based on the input information and generated information and lists the product on the platform.

[0067] These steps allow users to list products easily and hassle-free. The system automatically generates highly accurate listing information and smoothly connects it to the listing platform.

[0068] The processing flow will be explained below.

[0069] Step 1:

[0070] Users take a photo of the item they want to sell using their smartphone or camera.

[0071] Specific operation: A user takes a photo with a Canon EOS 80D camera.

[0072] Step 2:

[0073] The terminal uploads the photographed product image to the system server.

[0074] Specific operation: Send an image file to the server via a smartphone app.

[0075] Step 3:

[0076] The server receives the uploaded image data.

[0077] Specific operation: An image file is saved in the server's storage.

[0078] Step 4:

[0079] The server analyzes the product images using image recognition technology.

[0080] Specific operation: Runs an OCR algorithm to extract text information such as "Canon EOS 80D" from the image.

[0081] Step 5:

[0082] Based on the analysis results, the server searches an existing product database to obtain product information.

[0083] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[0084] Step 6:

[0085] The server automatically generates listing information based on the product information acquired.

[0086] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[0087] Step 7:

[0088] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[0089] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[0090] Step 8:

[0091] The user enters any necessary information and performs a final check.

[0092] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[0093] Step 9:

[0094] The terminal transmits the inputted completion information to the server.

[0095] Specific operation: Data including information entered by the user is sent to the server.

[0096] Step 10:

[0097] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[0098] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[0099] Step 11:

[0100] The server notifies the user that the listing procedure is complete.

[0101] Specific operation: Notify the user of completion via email or in-app notification.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] The process of listing items at conventional auctions and flea markets involves a lot of manual work, requiring time and effort. In particular, the process of taking product images, uploading the images, identifying product information, completing the information, and completing the listing procedure is complicated and places a heavy burden on users. The present invention aims to solve these problems and provide a system that allows users to easily list items.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes means for acquiring product images, means for uploading the acquired images to the server, means for analyzing the uploaded images and identifying product information using image recognition technology and optical character recognition technology (OCR), means for searching an existing product database based on the identified product information to acquire product information, means for automatically generating listing information based on the acquired product information, means for prompting a user to input missing listing information, and means for listing the listing information on a predetermined platform. This enables users to list products easily and quickly without hassle.

[0107] "Product images" are photographs or illustrations that contain visual information showing the product.

[0108] The "acquisition means" refers to hardware or software for receiving product images from users and importing them into the system.

[0109] "Uploading means" refers to hardware or software that has the function of transmitting product images from a terminal to a server.

[0110] "Means for analyzing" refers to software that includes image recognition technology and optical character recognition (OCR) technology for analyzing uploaded product images and extracting product information.

[0111] "Product information" refers to information necessary to identify a product, and includes, for example, the product name, model number, and features.

[0112] An "existing product database" is a database that contains pre-registered product information (JAN code, ISBN code, product name, specifications, images, etc.).

[0113] "Product Information" is detailed information about a particular product retrieved from an existing product database.

[0114] "Means for automatic generation" refers to software that has the function of automatically creating initial settings for listing titles, product descriptions, and prices based on acquired product information.

[0115] "Missing listing information" is data that is necessary to list an item but is missing from the automatically generated information, such as the item's condition or accessories.

[0116] The "means for inputting" refers to hardware or software that provides an interface for users to provide the missing commodity information to the system.

[0117] "Means for listing" refers to hardware or software that has the functionality to send all listing information to a designated platform and list products.

[0118] "Platform" means an online marketplace for listing, buying and selling products.

[0119] The present invention is a system that aims to significantly reduce the effort required when putting up an item at an auction or flea market. Specific embodiments of the present invention will be described below.

[0120] 1. Obtaining and uploading product images

[0121] A user takes a photo of the product they want to sell using a smartphone or digital camera. This captures a "product image." The user then uses an application on their device to upload the product image to the system. The device uses an HTTP POST request to send the image to the server.

[0122] 2. Image analysis and product information extraction

[0123] The server analyzes the received product images. For the analysis, it uses image recognition technology such as Google Cloud Vision API. This technology is used to analyze the information in the image and identify the product. It also uses OCR technology such as Tesseract to extract text information from the image. This allows it to obtain "product information" such as the product name, model number, and features.

[0124] 3. Obtaining product information

[0125] The server searches an existing product database based on the extracted product information. The product database stores, for example, JAN codes, ISBN codes, product names, specifications, images, etc. The server searches this database using SQL queries to retrieve the relevant product information.

[0126] 4. Automatic generation of listing information

[0127] The server automatically generates listing titles, product descriptions, and initial price settings based on the acquired product information. Templates are used for generation, and include titles such as "Canon EOS 80D Digital SLR Camera Body" and descriptions such as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." This generated information is sent to the device and displayed to the user.

[0128] 5. Information Complementary Interface

[0129] The terminal provides the user with an interface for entering missing listing information (such as product condition and accessories). The user accesses this interface and enters the necessary information. For example, the user can select or enter the product condition as "almost new" and the accessories as "battery, charger."

[0130] 6. Completing the listing process

[0131] The server integrates all the information entered by the user and the automatically generated information, and performs the final process to list the product on the listing platform. This process is performed using the platform's API, such as eBay API or Mercari API. The server notifies the user that the listing was successful.

[0132] Specific examples

[0133] For example, if a user wants to sell a Canon EOS 80D camera:

[0134] 1. The user takes a photo with the camera on their smartphone.

[0135] 2. Upload the images taken by the device to the system server.

[0136] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using the Google Cloud Vision API.

[0137] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[0138] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[0139] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[0140] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[0141] 8. The server lists the product on the designated platform based on the input information and generated information.

[0142] Prompt Sentence Examples

[0143] Prompt: "Please explain the system that automatically identifies product information and generates listing information after uploading product images. Please include the names of the specific hardware and software used, as well as the type of data processing and calculations performed."

[0144] This allows users to list products easily and quickly without any hassle. The system utilizes generative AI models to automatically generate highly accurate listing information and smoothly connect it to the listing platform.

[0145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0146] Step 1: Get product images

[0147] Users take a photo of the item they want to sell using a smartphone or digital camera.

[0148] Specifically, the user opens the camera app on their smartphone, frames the product, and presses the capture button.

[0149] Input: Physical Goods

[0150] Output: Digital image file

[0151] Step 2: Upload an image

[0152] The terminal uploads the captured product images to the system.

[0153] The user selects the image they have taken on the application and presses the upload button.

[0154] The device sends an HTTP POST request to the server to upload the image.

[0155] Input: Digital image file

[0156] Output: Image data sent to the server

[0157] Step 3: Image recognition and product identification

[0158] The server receives the uploaded images and performs analysis.

[0159] The server uses the Google Cloud Vision API to perform image recognition.

[0160] The server uses OCR technology (e.g., Tesseract) to extract the text in the image.

[0161] Input: Image data sent to the server

[0162] Output: Product information data such as product name, model number, and features

[0163] Step 4: Obtain product information

[0164] The server searches an existing product database based on the acquired product information.

[0165] The server generates an SQL query to search the product database.

[0166] Obtain relevant product information (e.g., JAN code, product specifications, product name, etc.).

[0167] Input: Product information data

[0168] Output: Detailed product information (JAN code, product specifications, product name, etc.)

[0169] Step 5: Auto-generate listings

[0170] Based on the acquired product information, the server automatically generates initial settings for the listing title, product description, and price.

[0171] Use templates to combine information and generate it automatically.

[0172] The generated listing information is sent to the terminal and displayed to the user.

[0173] Input: Product details

[0174] Output: Listing title, product description, price information

[0175] Step 6: Information Completion Interface

[0176] The terminal provides an interface for the user to input missing listing information (e.g., product condition, accessories).

[0177] Displays text boxes and drop-down menus for users to enter or select information.

[0178] Input: Information entered by the user

[0179] Output: Added listing information (item condition, accessories, etc.)

[0180] Step 7: Complete your listing

[0181] The server generates the final listing information based on all the entered information and automatically lists it on the listing platform.

[0182] Use APIs (e.g., eBay API, Mercari API) to send listing information to the platform.

[0183] If the listing is successful, the server sends a notification to the user.

[0184] Input: Final listing information

[0185] Output: Listing completion notification to the listing platform

[0186] In this way, the system supports users in putting items up for sale efficiently and quickly, significantly reducing the amount of work required by users.

[0187] (Application example 1)

[0188] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0189] Listing items on traditional auctions and flea markets requires a lot of time and effort. In particular, the process of uploading product images and manually entering the necessary information is cumbersome and burdensome for users. Setting an appropriate price for an item also requires knowledge and experience, making it difficult for beginners. Furthermore, linking listing information is often inconvenient, making it difficult to achieve a smooth listing process.

[0190] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0191] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images to identify product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, means for proposing a price based on conditions entered by the user, and means for automatically linking the generated listing information to a predetermined e-commerce platform. This allows users to easily list products and automatically proposes an appropriate price, making it possible for even beginners to efficiently complete the listing procedure.

[0192] "Product images" refer to image data of products that users wish to sell, taken using a digital camera or smartphone.

[0193] "Means for analyzing images and identifying product information" refers to technology that uses image recognition technology to automatically extract and identify information such as product names and features from acquired product images.

[0194] The "means for automatically generating listing information" is a technology that automatically generates listing titles, descriptions, prices, etc. based on identified product information.

[0195] The "means for allowing the user to input missing listing information" is a technique for providing an interface for the user to manually complete missing listing information when the automatically generated listing information is missing.

[0196] The "means for listing the listing information on a predetermined platform" refers to a technology for automatically uploading the created listing information to an e-commerce platform and completing the listing procedure.

[0197] The "price suggestion method" is a technology that automatically calculates an appropriate selling price for a product based on the product's condition and accessory information entered by the user.

[0198] An "e-commerce platform" is an online marketplace for buying and selling goods.

[0199] The present invention relates to a smartphone application that automatically identifies product information and generates listing information by taking a photo of a product and uploading the image. A specific embodiment of this system will be described below.

[0200] Image Acquisition and Upload:

[0201] Users take pictures of the items they want to sell using their smartphone camera, and the images are uploaded to the server via the application.

[0202] Product Identification:

[0203] The server receives the uploaded image data and identifies product information using image recognition technology, using libraries such as Pillow (PIL) and pytesseract. The server extracts product names, model numbers, features, etc. from the product images.

[0204] Get product information:

[0205] Based on the identified product information, the server uses an external database API to retrieve detailed product information, including JAN codes, product names, specifications, and additional images.

[0206] Auto-generated listings:

[0207] The server automatically generates listing information based on the acquired product information and the product condition and accessory information entered by the user. The generated listing information includes the listing title, description, and initial price setting. To propose a price, the server calculates an appropriate price based on the information entered by the user.

[0208] Listing information integration:

[0209] The final listing information is automatically uploaded from the server via the API of the e-commerce platform, allowing users to list their products without going through complicated procedures.

[0210] Hardware and software used:

[0211] Smartphone: A device that takes product images and runs applications.

[0212] Server: A central server for image analysis and data processing.

[0213] Pillow (PIL): An image processing library.

[0214] pytesseract: An OCR (character recognition) library.

[0215] requests: A library for making HTTP requests.

[0216] External image recognition API: An API that analyzes product images and retrieves product information.

[0217] External Database API: API for retrieving product information.

[0218] Examples:

[0219] For example, if a user wants to sell a particular camera, they might use the following prompt:

[0220] Example prompt sentence:

[0221] Please analyze the product images below and identify the product name, features, condition, and accessories.

[0222] Then, generate the listing title, description, and price, and create a JSON file to upload to the listing platform.

[0223] Product image path: "path / to / product_image.jpg"

[0224] User input information:

[0225] Condition: "Like New"

[0226] Accessories: "Battery, charger"

[0227] Output format:

[0228] {

[0229] "title": "<listing title>",

[0230] "description": "<description>",

[0231] "price": <price>

[0232] }

[0233] By inputting this prompt sentence into the generative AI model, the entire process from identifying product information to automatically generating listing information can be carried out efficiently.

[0234] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0235] Step 1:

[0236] The user takes a picture of the product using the smartphone camera.

[0237] Input: Product image

[0238] Output: Photographed product image data

[0239] Specific operation: The user takes a picture of the item they want to sell and saves it on their smartphone.

[0240] Step 2:

[0241] The product images taken by the terminal are uploaded to the server.

[0242] Input: Photographed product image data

[0243] Output: Product image data uploaded to the server

[0244] Specific operation: An application on the smartphone sends the captured image file to the server.

[0245] Step 3:

[0246] The server analyzes the uploaded image data and identifies the product information.

[0247] Input: Product image data uploaded to the server

[0248] Output: Identified product information

[0249] How it works: The server uses Pillow and pytesseract to analyze the image and identify the product name and model number. For example, it uses image recognition technology to extract text information about the product.

[0250] Step 4:

[0251] The server acquires product information from an external database based on the identified product information.

[0252] Input: Identified product information

[0253] Output: Product information (e.g. JAN code, product name, specifications, additional images)

[0254] Specific operation: The server uses the recognized product name and model number as a key to call an external database API and obtain detailed product information.

[0255] Step 5:

[0256] Listing information is automatically generated based on the product information acquired by the server.

[0257] Input: Product information and user-entered information (product condition and accessories)

[0258] Output: Auto-generated listing information (title, description, price)

[0259] Specific operation: The server combines the acquired product information with the user's input information to generate an listing title, description, and initial price. For example, the description may reflect the product's unique characteristics and the product's condition input by the user.

[0260] Step 6:

[0261] The server uploads the listing information to a designated e-commerce platform.

[0262] Input: Auto-generated listing information

[0263] Output: Listing information uploaded to e-commerce platform

[0264] Specific operation: The server automatically sends the generated listing information to the API of the e-commerce platform to complete the listing process, for example, by converting the listing information into JSON format and sending it to the API.

[0265] By following the above steps, users can easily list items for sale and efficiently complete the listing procedure.

[0266] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0267] The present invention is a system that makes the listing process more efficient and user-friendly by combining an emotion engine that recognizes and analyzes the emotions of users when they list items at auctions or flea markets. Specific embodiments of this system are described below.

[0268] Overall system overview

[0269] 1. Product image acquisition

[0270] Users take a photo of the product they want to sell using a smartphone or digital camera.

[0271] The product images taken by the terminal are uploaded to the system.

[0272] 2. Image Recognition and Product Identification

[0273] The server receives the uploaded image and identifies the product using image recognition technology.

[0274] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[0275] 3. Obtaining product information

[0276] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[0277] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[0278] 4. Automatic generation of listing information

[0279] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[0280] The terminal displays the automatically generated listing information to the user.

[0281] 5. Emotion recognition

[0282] The device uses an emotion engine to analyze the user's facial expressions, voice, etc., and recognizes the user's emotional state.

[0283] The server receives the emotion recognition results and dynamically adjusts the listing information and user interface if the user is dissatisfied or has questions.

[0284] 6. Information Complementary Interface

[0285] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[0286] The user enters the requested information and provides final confirmation.

[0287] 7. Completing the listing process

[0288] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[0289] The server uses the listing platform's API to complete the listing process.

[0290] Specific examples

[0291] For example, if a user wants to sell a Canon EOS 80D camera:

[0292] 1. The user takes a photo with the camera on their smartphone.

[0293] 2. Upload the images taken by the device to the system server.

[0294] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[0295] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[0296] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[0297] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[0298] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[0299] 8. The device analyzes the user's facial expressions and voice, and if the user feels anxious or has questions, displays additional assistance messages or tooltips.

[0300] 9. The server processes the listing based on the input information and generated information and lists the product on the platform.

[0301] In this way, by combining emotion engines, interactions can be provided that correspond to the user's emotional state, improving the overall user experience of the listing process.

[0302] The processing flow will be explained below.

[0303] Step 1:

[0304] Users take a photo of the item they want to sell using their smartphone or camera.

[0305] Specific action: A user takes a photo with a Canon EOS 80D camera.

[0306] Step 2:

[0307] The terminal uploads the photographed product image to the system server.

[0308] Specific operation: Send an image file to the server via a smartphone app.

[0309] Step 3:

[0310] The server receives the uploaded image data.

[0311] Specific operation: An image file is saved in the server's storage.

[0312] Step 4:

[0313] The server analyzes the product images using image recognition technology.

[0314] Specific operation: Run the OCR algorithm and extract the text information "Canon EOS 80D" from the image.

[0315] Step 5:

[0316] Based on the analysis results, the server searches an existing product database to obtain product information.

[0317] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[0318] Step 6:

[0319] The server automatically generates listing information based on the product information acquired.

[0320] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[0321] Step 7:

[0322] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[0323] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[0324] Step 8:

[0325] The device uses an emotion engine that analyzes the user's facial expressions and voice and recognizes the user's emotions.

[0326] Specific operation: The device's camera and microphone are used to analyze the user's facial expressions and tone of voice to assess their emotional state.

[0327] Step 9:

[0328] The server receives the emotion recognition results and adjusts the interface and feedback content according to the user's emotions.

[0329] Specific action: If the user feels anxious, display a more detailed guide or assistance message.

[0330] Step 10:

[0331] The user enters any necessary information and performs a final check.

[0332] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[0333] Step 11:

[0334] The terminal transmits the inputted completion information to the server.

[0335] Specific operation: Data including information entered by the user is sent to the server.

[0336] Step 12:

[0337] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[0338] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[0339] Step 13:

[0340] The server notifies the user that the listing procedure is complete.

[0341] Specific operation: Notify the user of completion via email or in-app notification.

[0342] Example 2

[0343] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] The process of listing items at traditional auctions and flea markets is often time-consuming and complicated for users. In particular, the time required to enter listing information and confirm detailed product information degrades the user experience. Furthermore, a one-sided interface that ignores the user's emotional state can cause anxiety and uncertainty. This creates a challenge for users, as it hinders the overall listing process.

[0345] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring product images using an imaging device, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for recognizing the user's emotional state using an emotion engine and dynamically adjusting the interface, means for prompting the user to input missing listing information, and means for listing the generated listing information on a predetermined network platform. This enables the user to list products easily and efficiently, and a more comfortable user experience can be provided by adjusting the interface according to the user's emotional state.

[0346] A "photography device" is a device that a user uses to take product images, such as a smartphone camera or a digital camera.

[0347] The "means for acquiring product images" refers to a means by which a user takes a product image using a photographing device and imports the image into the system.

[0348] "Means for analyzing images and identifying product information" refers to means for analyzing product images acquired by the system and extracting product information (e.g., product name, model number, etc.) using image recognition technology or optical character recognition (OCR).

[0349] "Means for automatically generating listing information" refers to means by which the system automatically generates listing information such as listing title, product description, and price based on the identified product information.

[0350] An "emotion engine" refers to an algorithm or software that analyzes a user's facial expressions and voice to recognize their emotional state.

[0351] "Means for recognizing emotional states and dynamically adjusting the interface" refers to a means for using an emotion engine to know the user's emotional state and adjusting the user interface in real time according to the results.

[0352] The "means for allowing the user to input missing information for auction" refers to a means for providing an interface for allowing the user to input information when information necessary for auction is missing.

[0353] "Means for listing the generated listing information on a specified network platform" refers to means for listing a product on a network platform (e.g., an auction site or a flea market app) using the automatically generated listing information.

[0354] This invention is a system that combines an emotion engine to make the listing process efficient and user-friendly when users list items at auctions or flea markets. The system is composed of a user, a terminal, and a server.

[0355] First, a user uses a camera such as a smartphone or digital camera to take a picture of the product they want to sell. The device then uploads the captured product image to the system. The server then receives the image and identifies the product information using image analysis and optical character recognition (OCR) technology. Specifically, the server uses an image analysis library (e.g., OpenCV) and OCR technology to extract features from the product image and read text information.

[0356] Next, the server searches an existing product database based on the acquired product information to obtain detailed product information (e.g., JAN code, product specifications, etc.), issues a database query, and temporarily stores the relevant product information.

[0357] Based on this information, the server automatically generates listing information, including an automatically generated listing title, product description, and default price, using a natural language processing model (e.g., GPT-3). The generated information is sent to the device and displayed to the user.

[0358] The emotion engine then analyzes the user's emotional state. The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The server receives the emotion analysis results and dynamically adjusts the user interface if the user is dissatisfied or has questions. For example, if the user has questions, it displays additional explanations or help messages.

[0359] For any missing listing information, the terminal provides an interface for the user to input the information. Specifically, an input form for the product's condition and accessories is displayed, and the user enters the information and makes a final confirmation. The input information is confirmed by pressing the final confirmation button.

[0360] Finally, the server generates the final listing information based on all the entered information and automatically lists the item on the designated listing platform. It sends the data to the listing platform using an API to complete the listing process. It receives a response from the listing platform and notifies the user that the listing has been completed.

[0361] As a concrete example, consider a user who wants to sell a Canon EOS 80D camera. The user first takes a photo of the camera with their smartphone and uploads it to the server. The server analyzes the image and identifies it as a "Canon EOS 80D" using OCR technology. The server then searches an existing product database to retrieve accurate product information. The server then automatically generates a listing title such as "Canon EOS 80D Digital SLR Camera Body" and a description such as "High-Performance Digital SLR Camera," which the device displays to the user. The user then enters additional information, and the emotion engine adjusts the interface to address the user's concerns and questions. Finally, the server sends all listing information to the network platform, completing the listing.

[0362] Example prompt sentence:

[0363] Please explain the process of analyzing photos taken by a user with a "digital camera," displaying an automatically generated listing title and description based on detailed product information, analyzing the user's facial expressions and voice, providing assistance messages according to their emotions, and finally listing the product on a designated listing platform.

[0364] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0365] Step 1: Get product images

[0366] The user takes a photo of the product they want to sell using a photographic device (e.g., a smartphone or digital camera).

[0367] Input: Product image

[0368] The terminal will upload the captured product image to the system. At this time, the terminal will display the message "Uploading image."

[0369] Output: Product images uploaded to the system

[0370] Step 2: Image recognition and product identification

[0371] The server receives the uploaded product images.

[0372] Input: Uploaded product image

[0373] The server uses an image analysis library (e.g., OpenCV) and optical character recognition (OCR) technology to analyze product images and extract product information (e.g., product name, model number, etc.).

[0374] Data processing: Extract features from images and recognize text information.

[0375] Output: Identified product information (e.g. Canon EOS 80D)

[0376] Step 3: Obtain product information

[0377] The server searches an existing product database based on the identified product information.

[0378] Input: Identified product information

[0379] The server issues a product database query to retrieve relevant product information (e.g., JAN code, product specifications, etc.).

[0380] Data Calculation: Information retrieval and results retrieval from product databases.

[0381] Output: Detailed product information (e.g., JAN code, product specifications)

[0382] Step 4: Auto-generate listings

[0383] The server automatically generates initial listing titles, product descriptions, and prices based on the acquired product information.

[0384] Input: Detailed product information

[0385] The server generates listing information using a natural language processing model (e.g., GPT-3).

[0386] Data processing: Listing information is created using a natural language generation model based on product information.

[0387] Output: Auto-generated listing title, description, and pricing

[0388] Step 5: Emotion Recognition

[0389] The device analyzes the user's facial expressions and voice using an emotion engine.

[0390] Input: User's facial expression data, voice data

[0391] The device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time.

[0392] Data processing: Identifying emotional states using facial expression and voice analysis algorithms.

[0393] Output: User's emotional state (e.g., anxiety, doubt)

[0394] Step 6: Dynamically Adjusting the Interface

[0395] The server dynamically adjusts the user interface based on the results of the emotion recognition.

[0396] Input: User's emotional state

[0397] The device will change its interface based on the emotion engine's analysis results, and will display additional explanations and help messages as needed.

[0398] Data manipulation: Executes logic to dynamically change interface elements based on the user's emotional state.

[0399] Output: A dynamically adjusted user interface

[0400] Step 7: Information Completion Interface

[0401] The terminal provides an interface for the user to input missing listing information.

[0402] Enter: Missing listing information items

[0403] The terminal displays a form where you can enter information about the product's condition and accessories.

[0404] Data Entry: The user enters the required information into a form and submits it to the system.

[0405] Output: Additional listing information entered

[0406] Step 8: Complete your listing

[0407] The server generates the final listing information based on all the information entered.

[0408] Input: Generated listing, additional information entered by the user

[0409] The server uses the listing platform's API to complete the listing process.

[0410] Data processing: Generate comprehensive listing information and complete listing via the network platform API.

[0411] Output: Listing completion notification, response from the listing platform

[0412] (Application example 2)

[0413] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0414] Conventionally, product and part registration and management processes have often been performed manually, resulting in inefficiency and stress for users. Furthermore, because the system does not take user feelings into consideration, it is prone to operational errors and registration errors. The present invention aims to solve these problems and provide an efficient and user-friendly registration system.

[0415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0416] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, and means for recognizing a user's emotions and dynamically adjusting information input and an interface according to the emotions. This makes it possible to identify product information using image recognition technology and adjust the interface according to the user's emotions.

[0417] "Product image" is image data of the product or part that the user wishes to register or list.

[0418] "Means for acquiring" refers to a method or device that allows a user to take product images using a camera or smartphone and upload them to the system.

[0419] The "analysis means" refers to the technology or method used to analyze the acquired product image and identify product information and part information.

[0420] "Product information" is detailed data including the model number, name, and features of a product or part.

[0421] "Listing Information" means information displayed on the listing platform, such as the listing title, description, and price.

[0422] The "means for automatic generation" refers to a technique or method by which the system automatically creates listing information based on the identified product information.

[0423] "Means for input" refers to the interface or process by which users can complete missing listing information and input it into the system.

[0424] A "prescribed platform" refers to an online service such as an auction site or flea market app where products or parts are actually listed.

[0425] "Means for recognizing emotions" refers to technologies and methods for analyzing a user's facial expressions and voice to determine their emotional state.

[0426] "Dynamic adjustment" refers to a method for changing the interface or information input process in real time according to the user's emotional state.

[0427] This invention is a system that combines an emotion engine to streamline the process and provide a user-friendly interface when a user registers or lists a product or part. Specific embodiments of this system are described below.

[0428] Hardware and software used

[0429] 1. Hardware:

[0430] Take a photo of the product using your smartphone camera or a dedicated camera.

[0431] A face detection camera is used to capture the user's facial expressions.

[0432] 2. Software:

[0433] Image analysis technology: Using OpenCV and Tesseract, product images are analyzed to identify product information.

[0434] Database Access: Uses DatabaseConnector to retrieve details from the product database based on the identified product information.

[0435] Emotion Recognition: Use the EmotionRecognizer library to recognize the user's emotional state by analyzing their facial expressions and voice.

[0436] API communication: Use APIConnector to register information on the designated listing platform.

[0437] Specific examples

[0438] The following processing procedure will explain a specific example of when a user registers a part.

[0439] Program processing

[0440] 1. Obtaining product images:

[0441] Users take a photo of the part using a smartphone or dedicated camera and upload the image to the system.

[0442] 2. Image analysis and product information identification:

[0443] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information, which is then used to identify the part.

[0444] 3. Get part information:

[0445] Based on the identified part information, the server uses the DatabaseConnector to retrieve detailed part information from an existing product database, including part number, name, characteristics, etc.

[0446] 4. View automatically generated registration information:

[0447] Based on the acquired part information, the server automatically generates registration information such as registration title, description, and initial inventory, and displays it to the user.

[0448] 5. Emotion recognition:

[0449] It uses a face detection camera to capture the user's facial expressions and analyzes their emotional state using the EmotionRecognizer library, and the analysis results are sent to the server.

[0450] 6. Dynamic Adjustment:

[0451] The server dynamically adjusts information input and the interface depending on the user's emotional state, displaying additional assistance messages and tooltips if the user feels anxious or uncertain.

[0452] 7. Completion and final confirmation:

[0453] The user enters any missing information (e.g., arrival date and part condition) and performs a final check.

[0454] 8. Listing Information:

[0455] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[0456] Prompt Sentence Examples

[0457] Below is an example of a prompt that can be used to register a part:

[0458] Prompt 1:

[0459] To register a part, take a photo of the part and upload it. Once the photo is uploaded, the system will recognize the part and automatically generate the registration information. If you have any problems or questions during the process, an assistance message will be displayed, so please follow the instructions.

[0460] response:

[0461] The part was successfully recognized. The following information was automatically generated:

[0462] Title: Part Name

[0463] Description: Part description

[0464] Initial stock: 1

[0465] Please enter any missing information (such as arrival date) and make a final confirmation.

[0466] This allows the interface to adapt to the user's emotional state, making the registration process more efficient and stress-free.

[0467] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0468] Step 1:

[0469] Users take photos of products or parts using a smartphone or dedicated camera and upload the images to the system.

[0470] Input: Product image taken by the user

[0471] Output: Product image data uploaded to the system

[0472] Specific operation: The user launches the camera app, takes a picture of the product, and then sends the image file to the server using the specified upload button.

[0473] Step 2:

[0474] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information from the product image.

[0475] Input: Uploaded product image data

[0476] Output: Text information extracted using OCR technology (e.g., part number, product name)

[0477] Specific operation: The server reads the image file, preprocesses it with OpenCV, and extracts text using Tesseract.

[0478] Step 3:

[0479] The server uses the DatabaseConnector to search an existing product database based on the extracted character information and obtains detailed part information.

[0480] Input: Character information extracted using OCR technology

[0481] Output: Detailed part information (e.g. part name, features, specs) retrieved from product database

[0482] Specific operation: The server uses the text information as a search query and DatabaseConnector to retrieve data on related parts from the database.

[0483] Step 4:

[0484] Based on the part information acquired by the server, registration information such as registration title, description, and initial inventory is automatically generated and sent to the terminal.

[0485] Input: Detailed part information retrieved from the database

[0486] Output: Auto-generated registration information

[0487] Specific operation: The server analyzes the part information, automatically formats the necessary registration information (title, description, inventory, etc.) using a template, and sends it to the user's device.

[0488] Step 5:

[0489] The terminal displays the automatically generated registration information to the user and prompts the user to enter any missing information as necessary.

[0490] Input: Auto-generated registration information sent from the server

[0491] Output: Final registration information including any supplementary information entered by the user

[0492] Specific operation: The device displays the registration information on the screen and provides a format (e.g., text entry fields) in which the user can enter any missing information.

[0493] Step 6:

[0494] It uses a face detection camera to capture the user's facial expressions, analyzes their emotional state using the EmotionRecognizer library, and sends the analysis results to the server.

[0495] Input: User facial expression capture image

[0496] Output: User's emotional state data

[0497] Specific operation: The device's camera periodically captures the user's facial expressions, inputs the data into the EmotionRecognizer library, analyzes the emotional state, and sends it to the server.

[0498] Step 7:

[0499] The server dynamically adjusts information input and interface according to the user's emotional state.

[0500] Input: User's emotional state data

[0501] Output: Dynamically adjusted interface or assist message

[0502] Specific operation: The server analyzes the received emotional state data, and if the user feels anxious or uncertain, it generates additional assistance messages or tooltips and displays them on the device.

[0503] Step 8:

[0504] The user enters any missing information and performs a final check.

[0505] Input: Missing information (e.g., arrival date, part condition)

[0506] Output: Final verified complete registration information

[0507] Specific operation: The user enters the required information and presses the final confirmation button, which sends the information to the server.

[0508] Step 9:

[0509] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[0510] Input: Complete and final registration information

[0511] Output: Notification of completion of registration to the listing platform

[0512] What happens: The server consolidates all the information, uses the APIConnector to send the necessary data to the listing platform's API, and generates a notification to complete the registration process.

[0513] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0514] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0515] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0516] [Second embodiment]

[0517] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0518] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0519] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0520] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0521] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0522] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0523] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0524] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0525] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0526] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0527] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0528] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0529] The present invention is a system that significantly reduces the effort required to put up an item at an auction or flea market. A user photographs an item and uploads the image to the system, which then automatically identifies the item information and generates listing information. Specific embodiments of this system are described below.

[0530] Overall system overview

[0531] 1. Product image acquisition

[0532] Users take a photo of the product they want to sell using a smartphone or digital camera.

[0533] The product images taken by the terminal are uploaded to the system.

[0534] 2. Image Recognition and Product Identification

[0535] The server receives the uploaded image and identifies the product using image recognition technology.

[0536] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[0537] 3. Obtaining product information

[0538] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[0539] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[0540] 4. Automatic generation of listing information

[0541] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[0542] The terminal displays the automatically generated listing information to the user.

[0543] 5. Information Complementary Interface

[0544] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[0545] The user enters the requested information and provides final confirmation.

[0546] 6. Completing the listing process

[0547] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[0548] The server uses the listing platform's API to complete the listing process.

[0549] Specific examples

[0550] For example, if a user wants to sell a Canon EOS 80D camera:

[0551] 1. The user takes a photo with the camera on their smartphone.

[0552] 2. Upload the images taken by the device to the system server.

[0553] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[0554] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[0555] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[0556] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[0557] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[0558] 8. The server processes the listing based on the input information and generated information and lists the product on the platform.

[0559] These steps allow users to list products easily and hassle-free. The system automatically generates highly accurate listing information and smoothly connects it to the listing platform.

[0560] The processing flow will be explained below.

[0561] Step 1:

[0562] Users take a photo of the item they want to sell using their smartphone or camera.

[0563] Specific operation: A user takes a photo with a Canon EOS 80D camera.

[0564] Step 2:

[0565] The terminal uploads the photographed product image to the system server.

[0566] Specific operation: Send an image file to the server via a smartphone app.

[0567] Step 3:

[0568] The server receives the uploaded image data.

[0569] Specific operation: An image file is saved in the server's storage.

[0570] Step 4:

[0571] The server analyzes the product images using image recognition technology.

[0572] Specific operation: Runs an OCR algorithm to extract text information such as "Canon EOS 80D" from the image.

[0573] Step 5:

[0574] Based on the analysis results, the server searches an existing product database to obtain product information.

[0575] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[0576] Step 6:

[0577] The server automatically generates listing information based on the product information acquired.

[0578] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[0579] Step 7:

[0580] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[0581] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[0582] Step 8:

[0583] The user enters any necessary information and performs a final check.

[0584] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[0585] Step 9:

[0586] The terminal transmits the inputted completion information to the server.

[0587] Specific operation: Data including information entered by the user is sent to the server.

[0588] Step 10:

[0589] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[0590] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[0591] Step 11:

[0592] The server notifies the user that the listing procedure is complete.

[0593] Specific operation: Notify the user of completion via email or in-app notification.

[0594] Example 1

[0595] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0596] The process of listing items at conventional auctions and flea markets involves a lot of manual work, requiring time and effort. In particular, the process of taking product images, uploading the images, identifying product information, completing the information, and completing the listing procedure is complicated and places a heavy burden on users. The present invention aims to solve these problems and provide a system that allows users to easily list items.

[0597] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0598] In this invention, the server includes means for acquiring product images, means for uploading the acquired images to the server, means for analyzing the uploaded images and identifying product information using image recognition technology and optical character recognition technology (OCR), means for searching an existing product database based on the identified product information to acquire product information, means for automatically generating listing information based on the acquired product information, means for prompting a user to input missing listing information, and means for listing the listing information on a predetermined platform. This enables users to list products easily and quickly without hassle.

[0599] "Product images" are photographs or illustrations that contain visual information showing the product.

[0600] The "acquisition means" refers to hardware or software for receiving product images from users and importing them into the system.

[0601] "Uploading means" refers to hardware or software that has the function of transmitting product images from a terminal to a server.

[0602] "Means for analyzing" refers to software that includes image recognition technology and optical character recognition (OCR) technology for analyzing uploaded product images and extracting product information.

[0603] "Product information" refers to information necessary to identify a product, and includes, for example, the product name, model number, and features.

[0604] An "existing product database" is a database that contains pre-registered product information (JAN code, ISBN code, product name, specifications, images, etc.).

[0605] "Product Information" is detailed information about a particular product retrieved from an existing product database.

[0606] "Means for automatic generation" refers to software that has the function of automatically creating initial settings for listing titles, product descriptions, and prices based on acquired product information.

[0607] "Missing listing information" is data that is necessary to list an item but is missing from the automatically generated information, such as the item's condition or accessories.

[0608] The "means for inputting" refers to hardware or software that provides an interface for users to provide the missing commodity information to the system.

[0609] "Means for listing" refers to hardware or software that has the functionality to send all listing information to a designated platform and list products.

[0610] "Platform" means an online marketplace for listing, buying and selling products.

[0611] The present invention is a system that aims to significantly reduce the effort required when putting up an item at an auction or flea market. Specific embodiments of the present invention will be described below.

[0612] 1. Obtaining and uploading product images

[0613] A user takes a photo of the product they want to sell using a smartphone or digital camera. This captures a "product image." The user then uses an application on their device to upload the product image to the system. The device uses an HTTP POST request to send the image to the server.

[0614] 2. Image analysis and product information extraction

[0615] The server analyzes the received product images. For the analysis, it uses image recognition technology such as Google Cloud Vision API. This technology is used to analyze the information in the image and identify the product. It also uses OCR technology such as Tesseract to extract text information from the image. This allows it to obtain "product information" such as the product name, model number, and features.

[0616] 3. Obtaining product information

[0617] The server searches an existing product database based on the extracted product information. The product database stores, for example, JAN codes, ISBN codes, product names, specifications, images, etc. The server searches this database using SQL queries to retrieve the relevant product information.

[0618] 4. Automatic generation of listing information

[0619] The server automatically generates listing titles, product descriptions, and initial price settings based on the acquired product information. Templates are used for generation, and include titles such as "Canon EOS 80D Digital SLR Camera Body" and descriptions such as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." This generated information is sent to the device and displayed to the user.

[0620] 5. Information Complementary Interface

[0621] The terminal provides the user with an interface for entering missing listing information (such as product condition and accessories). The user accesses this interface and enters the necessary information. For example, the user can select or enter the product condition as "almost new" and the accessories as "battery, charger."

[0622] 6. Completing the listing process

[0623] The server integrates all the information entered by the user and the automatically generated information, and performs the final process to list the product on the listing platform. This process is performed using the platform's API, such as eBay API or Mercari API. The server notifies the user that the listing was successful.

[0624] Specific examples

[0625] For example, if a user wants to sell a Canon EOS 80D camera:

[0626] 1. The user takes a photo with the camera on their smartphone.

[0627] 2. Upload the images taken by the device to the system server.

[0628] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using the Google Cloud Vision API.

[0629] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[0630] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[0631] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[0632] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[0633] 8. The server lists the product on the designated platform based on the input information and generated information.

[0634] Prompt Sentence Examples

[0635] Prompt: "Please explain the system that automatically identifies product information and generates listing information after uploading product images. Please include the names of the specific hardware and software used, as well as the type of data processing and calculations performed."

[0636] This allows users to list products easily and quickly without any hassle. The system utilizes generative AI models to automatically generate highly accurate listing information and smoothly connect it to the listing platform.

[0637] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0638] Step 1: Get product images

[0639] Users take a photo of the item they want to sell using a smartphone or digital camera.

[0640] Specifically, the user opens the camera app on their smartphone, frames the product, and presses the capture button.

[0641] Input: Physical Goods

[0642] Output: Digital image file

[0643] Step 2: Upload an image

[0644] The terminal uploads the captured product images to the system.

[0645] The user selects the image they have taken on the application and presses the upload button.

[0646] The device sends an HTTP POST request to the server to upload the image.

[0647] Input: Digital image file

[0648] Output: Image data sent to the server

[0649] Step 3: Image recognition and product identification

[0650] The server receives the uploaded images and performs analysis.

[0651] The server uses the Google Cloud Vision API to perform image recognition.

[0652] The server uses OCR technology (e.g., Tesseract) to extract the text in the image.

[0653] Input: Image data sent to the server

[0654] Output: Product information data such as product name, model number, and features

[0655] Step 4: Obtain product information

[0656] The server searches an existing product database based on the acquired product information.

[0657] The server generates an SQL query to search the product database.

[0658] Obtain relevant product information (e.g., JAN code, product specifications, product name, etc.).

[0659] Input: Product information data

[0660] Output: Detailed product information (JAN code, product specifications, product name, etc.)

[0661] Step 5: Auto-generate listings

[0662] Based on the acquired product information, the server automatically generates initial settings for the listing title, product description, and price.

[0663] Use templates to combine information and generate it automatically.

[0664] The generated listing information is sent to the terminal and displayed to the user.

[0665] Input: Product details

[0666] Output: Listing title, product description, price information

[0667] Step 6: Information Completion Interface

[0668] The terminal provides an interface for the user to input missing listing information (e.g., product condition, accessories).

[0669] Displays text boxes and drop-down menus for users to enter or select information.

[0670] Input: Information entered by the user

[0671] Output: Added listing information (item condition, accessories, etc.)

[0672] Step 7: Complete your listing

[0673] The server generates the final listing information based on all the entered information and automatically lists it on the listing platform.

[0674] Use APIs (e.g., eBay API, Mercari API) to send listing information to the platform.

[0675] If the listing is successful, the server sends a notification to the user.

[0676] Input: Final listing information

[0677] Output: Listing completion notification to the listing platform

[0678] In this way, the system supports users in putting items up for sale efficiently and quickly, significantly reducing the amount of work required by users.

[0679] (Application example 1)

[0680] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0681] Listing items on traditional auctions and flea markets requires a lot of time and effort. In particular, the process of uploading product images and manually entering the necessary information is cumbersome and burdensome for users. Setting an appropriate price for an item also requires knowledge and experience, making it difficult for beginners. Furthermore, linking listing information is often inconvenient, making it difficult to achieve a smooth listing process.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0683] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images to identify product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, means for proposing a price based on conditions entered by the user, and means for automatically linking the generated listing information to a predetermined e-commerce platform. This allows users to easily list products and automatically proposes an appropriate price, making it possible for even beginners to efficiently complete the listing procedure.

[0684] "Product images" refer to image data of products that users wish to sell, taken using a digital camera or smartphone.

[0685] "Means for analyzing images and identifying product information" refers to technology that uses image recognition technology to automatically extract and identify information such as product names and features from acquired product images.

[0686] The "means for automatically generating listing information" is a technology that automatically generates listing titles, descriptions, prices, etc. based on identified product information.

[0687] The "means for allowing the user to input missing listing information" is a technique for providing an interface for the user to manually complete missing listing information when the automatically generated listing information is missing.

[0688] The "means for listing the listing information on a predetermined platform" refers to a technology for automatically uploading the created listing information to an e-commerce platform and completing the listing procedure.

[0689] The "price suggestion method" is a technology that automatically calculates an appropriate selling price for a product based on the product's condition and accessory information entered by the user.

[0690] An "e-commerce platform" is an online marketplace for buying and selling goods.

[0691] The present invention relates to a smartphone application that automatically identifies product information and generates listing information by taking a photo of a product and uploading the image. A specific embodiment of this system will be described below.

[0692] Image Acquisition and Upload:

[0693] Users take pictures of the items they want to sell using their smartphone camera, and the images are uploaded to the server via the application.

[0694] Product Identification:

[0695] The server receives the uploaded image data and identifies product information using image recognition technology, using libraries such as Pillow (PIL) and pytesseract. The server extracts product names, model numbers, features, etc. from the product images.

[0696] Get product information:

[0697] Based on the identified product information, the server uses an external database API to retrieve detailed product information, including JAN codes, product names, specifications, and additional images.

[0698] Auto-generated listings:

[0699] The server automatically generates listing information based on the acquired product information and the product condition and accessory information entered by the user. The generated listing information includes the listing title, description, and initial price setting. To propose a price, the server calculates an appropriate price based on the information entered by the user.

[0700] Listing information integration:

[0701] The final listing information is automatically uploaded from the server via the API of the e-commerce platform, allowing users to list their products without going through complicated procedures.

[0702] Hardware and software used:

[0703] Smartphone: A device that takes product images and runs applications.

[0704] Server: A central server for image analysis and data processing.

[0705] Pillow (PIL): An image processing library.

[0706] pytesseract: An OCR (character recognition) library.

[0707] requests: A library for making HTTP requests.

[0708] External image recognition API: An API that analyzes product images and retrieves product information.

[0709] External Database API: API for retrieving product information.

[0710] Examples:

[0711] For example, if a user wants to sell a particular camera, they might use the following prompt:

[0712] Example prompt sentence:

[0713] Please analyze the product images below and identify the product name, features, condition, and accessories.

[0714] Then, generate the listing title, description, and price, and create a JSON file to upload to the listing platform.

[0715] Product image path: "path / to / product_image.jpg"

[0716] User input information:

[0717] Condition: "Like New"

[0718] Accessories: "Battery, charger"

[0719] Output format:

[0720] {

[0721] "title": "<listing title>",

[0722] "description": "<description>",

[0723] "price": <price>

[0724] }

[0725] By inputting this prompt sentence into the generative AI model, the entire process from identifying product information to automatically generating listing information can be carried out efficiently.

[0726] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0727] Step 1:

[0728] The user takes a picture of the product using the smartphone camera.

[0729] Input: Product image

[0730] Output: Photographed product image data

[0731] Specific operation: The user takes a picture of the item they want to sell and saves it on their smartphone.

[0732] Step 2:

[0733] The product images taken by the terminal are uploaded to the server.

[0734] Input: Photographed product image data

[0735] Output: Product image data uploaded to the server

[0736] Specific operation: An application on the smartphone sends the captured image file to the server.

[0737] Step 3:

[0738] The server analyzes the uploaded image data and identifies the product information.

[0739] Input: Product image data uploaded to the server

[0740] Output: Identified product information

[0741] How it works: The server uses Pillow and pytesseract to analyze the image and identify the product name and model number. For example, it uses image recognition technology to extract text information about the product.

[0742] Step 4:

[0743] The server acquires product information from an external database based on the identified product information.

[0744] Input: Identified product information

[0745] Output: Product information (e.g. JAN code, product name, specifications, additional images)

[0746] Specific operation: The server uses the recognized product name and model number as a key to call an external database API and obtain detailed product information.

[0747] Step 5:

[0748] Listing information is automatically generated based on the product information acquired by the server.

[0749] Input: Product information and user-entered information (product condition and accessories)

[0750] Output: Auto-generated listing information (title, description, price)

[0751] Specific operation: The server combines the acquired product information with the user's input information to generate an listing title, description, and initial price. For example, the description may reflect the product's unique characteristics and the product's condition input by the user.

[0752] Step 6:

[0753] The server uploads the listing information to a designated e-commerce platform.

[0754] Input: Auto-generated listing information

[0755] Output: Listing information uploaded to e-commerce platform

[0756] Specific operation: The server automatically sends the generated listing information to the API of the e-commerce platform to complete the listing process, for example, by converting the listing information into JSON format and sending it to the API.

[0757] By following the above steps, users can easily list items for sale and efficiently complete the listing procedure.

[0758] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0759] The present invention is a system that makes the listing process more efficient and user-friendly by combining an emotion engine that recognizes and analyzes the emotions of users when they list items at auctions or flea markets. Specific embodiments of this system are described below.

[0760] Overall system overview

[0761] 1. Product image acquisition

[0762] Users take a photo of the product they want to sell using a smartphone or digital camera.

[0763] The product images taken by the terminal are uploaded to the system.

[0764] 2. Image Recognition and Product Identification

[0765] The server receives the uploaded image and identifies the product using image recognition technology.

[0766] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[0767] 3. Obtaining product information

[0768] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[0769] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[0770] 4. Automatic generation of listing information

[0771] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[0772] The terminal displays the automatically generated listing information to the user.

[0773] 5. Emotion recognition

[0774] The device uses an emotion engine to analyze the user's facial expressions, voice, etc., and recognizes the user's emotional state.

[0775] The server receives the emotion recognition results and dynamically adjusts the listing information and user interface if the user is dissatisfied or has questions.

[0776] 6. Information Complementary Interface

[0777] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[0778] The user enters the requested information and provides final confirmation.

[0779] 7. Completing the listing process

[0780] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[0781] The server uses the listing platform's API to complete the listing process.

[0782] Specific examples

[0783] For example, if a user wants to sell a Canon EOS 80D camera:

[0784] 1. The user takes a photo with the camera on their smartphone.

[0785] 2. Upload the images taken by the device to the system server.

[0786] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[0787] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[0788] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[0789] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[0790] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[0791] 8. The device analyzes the user's facial expressions and voice, and if the user feels anxious or has questions, displays additional assistance messages or tooltips.

[0792] 9. The server processes the listing based on the input information and generated information and lists the product on the platform.

[0793] In this way, by combining emotion engines, interactions can be provided that correspond to the user's emotional state, improving the overall user experience of the listing process.

[0794] The processing flow will be explained below.

[0795] Step 1:

[0796] Users take a photo of the item they want to sell using their smartphone or camera.

[0797] Specific action: A user takes a photo with a Canon EOS 80D camera.

[0798] Step 2:

[0799] The terminal uploads the photographed product image to the system server.

[0800] Specific operation: Send an image file to the server via a smartphone app.

[0801] Step 3:

[0802] The server receives the uploaded image data.

[0803] Specific operation: An image file is saved in the server's storage.

[0804] Step 4:

[0805] The server analyzes the product images using image recognition technology.

[0806] Specific operation: Run the OCR algorithm and extract the text information "Canon EOS 80D" from the image.

[0807] Step 5:

[0808] Based on the analysis results, the server searches an existing product database to obtain product information.

[0809] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[0810] Step 6:

[0811] The server automatically generates listing information based on the product information acquired.

[0812] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[0813] Step 7:

[0814] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[0815] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[0816] Step 8:

[0817] The device uses an emotion engine that analyzes the user's facial expressions and voice and recognizes the user's emotions.

[0818] Specific operation: The device's camera and microphone are used to analyze the user's facial expressions and tone of voice to assess their emotional state.

[0819] Step 9:

[0820] The server receives the emotion recognition results and adjusts the interface and feedback content according to the user's emotions.

[0821] Specific action: If the user feels anxious, display a more detailed guide or assistance message.

[0822] Step 10:

[0823] The user enters any necessary information and performs a final check.

[0824] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[0825] Step 11:

[0826] The terminal transmits the inputted completion information to the server.

[0827] Specific operation: Data including information entered by the user is sent to the server.

[0828] Step 12:

[0829] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[0830] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[0831] Step 13:

[0832] The server notifies the user that the listing procedure is complete.

[0833] Specific operation: Notify the user of completion via email or in-app notification.

[0834] Example 2

[0835] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0836] The process of listing items at traditional auctions and flea markets is often time-consuming and complicated for users. In particular, the time required to enter listing information and confirm detailed product information degrades the user experience. Furthermore, a one-sided interface that ignores the user's emotional state can cause anxiety and uncertainty. This creates a challenge for users, as it hinders the overall listing process.

[0837] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring product images using an imaging device, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for recognizing the user's emotional state using an emotion engine and dynamically adjusting the interface, means for prompting the user to input missing listing information, and means for listing the generated listing information on a predetermined network platform. This enables the user to list products easily and efficiently, and a more comfortable user experience can be provided by adjusting the interface according to the user's emotional state.

[0838] A "photography device" is a device that a user uses to take product images, such as a smartphone camera or a digital camera.

[0839] The "means for acquiring product images" refers to a means by which a user takes a product image using a photographing device and imports the image into the system.

[0840] "Means for analyzing images and identifying product information" refers to means for analyzing product images acquired by the system and extracting product information (e.g., product name, model number, etc.) using image recognition technology or optical character recognition (OCR).

[0841] "Means for automatically generating listing information" refers to means by which the system automatically generates listing information such as listing title, product description, and price based on the identified product information.

[0842] An "emotion engine" refers to an algorithm or software that analyzes a user's facial expressions and voice to recognize their emotional state.

[0843] "Means for recognizing emotional states and dynamically adjusting the interface" refers to a means for using an emotion engine to know the user's emotional state and adjusting the user interface in real time according to the results.

[0844] The "means for allowing the user to input missing information for auction" refers to a means for providing an interface for allowing the user to input information when information necessary for auction is missing.

[0845] "Means for listing the generated listing information on a specified network platform" refers to means for listing a product on a network platform (e.g., an auction site or a flea market app) using the automatically generated listing information.

[0846] This invention is a system that combines an emotion engine to make the listing process efficient and user-friendly when users list items at auctions or flea markets. The system is composed of a user, a terminal, and a server.

[0847] First, a user uses a camera such as a smartphone or digital camera to take a picture of the product they want to sell. The device then uploads the captured product image to the system. The server then receives the image and identifies the product information using image analysis and optical character recognition (OCR) technology. Specifically, the server uses an image analysis library (e.g., OpenCV) and OCR technology to extract features from the product image and read text information.

[0848] Next, the server searches an existing product database based on the acquired product information to obtain detailed product information (e.g., JAN code, product specifications, etc.), issues a database query, and temporarily stores the relevant product information.

[0849] Based on this information, the server automatically generates listing information, including an automatically generated listing title, product description, and default price, using a natural language processing model (e.g., GPT-3). The generated information is sent to the device and displayed to the user.

[0850] The emotion engine then analyzes the user's emotional state. The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The server receives the emotion analysis results and dynamically adjusts the user interface if the user is dissatisfied or has questions. For example, if the user has questions, it displays additional explanations or help messages.

[0851] For any missing listing information, the terminal provides an interface for the user to input the information. Specifically, an input form for the product's condition and accessories is displayed, and the user enters the information and makes a final confirmation. The input information is confirmed by pressing the final confirmation button.

[0852] Finally, the server generates the final listing information based on all the entered information and automatically lists the item on the designated listing platform. It sends the data to the listing platform using an API to complete the listing process. It receives a response from the listing platform and notifies the user that the listing has been completed.

[0853] As a concrete example, consider a user who wants to sell a Canon EOS 80D camera. The user first takes a photo of the camera with their smartphone and uploads it to the server. The server analyzes the image and identifies it as a "Canon EOS 80D" using OCR technology. The server then searches an existing product database to retrieve accurate product information. The server then automatically generates a listing title such as "Canon EOS 80D Digital SLR Camera Body" and a description such as "High-Performance Digital SLR Camera," which the device displays to the user. The user then enters additional information, and the emotion engine adjusts the interface to address the user's concerns and questions. Finally, the server sends all listing information to the network platform, completing the listing.

[0854] Example prompt sentence:

[0855] Please explain the process of analyzing photos taken by a user with a "digital camera," displaying an automatically generated listing title and description based on detailed product information, analyzing the user's facial expressions and voice, providing assistance messages according to their emotions, and finally listing the product on a designated listing platform.

[0856] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0857] Step 1: Get product images

[0858] The user takes a photo of the product they want to sell using a photographic device (e.g., a smartphone or digital camera).

[0859] Input: Product image

[0860] The terminal will upload the captured product image to the system. At this time, the terminal will display the message "Uploading image."

[0861] Output: Product images uploaded to the system

[0862] Step 2: Image recognition and product identification

[0863] The server receives the uploaded product images.

[0864] Input: Uploaded product image

[0865] The server uses an image analysis library (e.g., OpenCV) and optical character recognition (OCR) technology to analyze product images and extract product information (e.g., product name, model number, etc.).

[0866] Data processing: Extract features from images and recognize text information.

[0867] Output: Identified product information (e.g. Canon EOS 80D)

[0868] Step 3: Obtain product information

[0869] The server searches an existing product database based on the identified product information.

[0870] Input: Identified product information

[0871] The server issues a product database query to retrieve relevant product information (e.g., JAN code, product specifications, etc.).

[0872] Data Calculation: Information retrieval and results retrieval from product databases.

[0873] Output: Detailed product information (e.g., JAN code, product specifications)

[0874] Step 4: Auto-generate listings

[0875] The server automatically generates initial listing titles, product descriptions, and prices based on the acquired product information.

[0876] Input: Detailed product information

[0877] The server generates listing information using a natural language processing model (e.g., GPT-3).

[0878] Data processing: Listing information is created using a natural language generation model based on product information.

[0879] Output: Auto-generated listing title, description, and pricing

[0880] Step 5: Emotion Recognition

[0881] The device analyzes the user's facial expressions and voice using an emotion engine.

[0882] Input: User's facial expression data, voice data

[0883] The device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time.

[0884] Data processing: Identifying emotional states using facial expression and voice analysis algorithms.

[0885] Output: User's emotional state (e.g., anxiety, doubt)

[0886] Step 6: Dynamically Adjusting the Interface

[0887] The server dynamically adjusts the user interface based on the results of the emotion recognition.

[0888] Input: User's emotional state

[0889] The device will change its interface based on the emotion engine's analysis results, and will display additional explanations and help messages as needed.

[0890] Data manipulation: Executes logic to dynamically change interface elements based on the user's emotional state.

[0891] Output: A dynamically adjusted user interface

[0892] Step 7: Information Completion Interface

[0893] The terminal provides an interface for the user to input missing listing information.

[0894] Enter: Missing listing information items

[0895] The terminal displays a form where you can enter information about the product's condition and accessories.

[0896] Data Entry: The user enters the required information into a form and submits it to the system.

[0897] Output: Additional listing information entered

[0898] Step 8: Complete your listing

[0899] The server generates the final listing information based on all the information entered.

[0900] Input: Generated listing, additional information entered by the user

[0901] The server uses the listing platform's API to complete the listing process.

[0902] Data processing: Generate comprehensive listing information and complete listing via the network platform API.

[0903] Output: Listing completion notification, response from the listing platform

[0904] (Application example 2)

[0905] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0906] Conventionally, product and part registration and management processes have often been performed manually, resulting in inefficiency and stress for users. Furthermore, because the system does not take user feelings into consideration, it is prone to operational errors and registration errors. The present invention aims to solve these problems and provide an efficient and user-friendly registration system.

[0907] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0908] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, and means for recognizing a user's emotions and dynamically adjusting information input and an interface according to the emotions. This makes it possible to identify product information using image recognition technology and adjust the interface according to the user's emotions.

[0909] "Product image" is image data of the product or part that the user wishes to register or list.

[0910] "Means for acquiring" refers to a method or device that allows a user to take product images using a camera or smartphone and upload them to the system.

[0911] The "analysis means" refers to the technology or method used to analyze the acquired product image and identify product information and part information.

[0912] "Product information" is detailed data including the model number, name, and features of a product or part.

[0913] "Listing Information" means information displayed on the listing platform, such as the listing title, description, and price.

[0914] The "means for automatic generation" refers to a technique or method by which the system automatically creates listing information based on the identified product information.

[0915] "Means for input" refers to the interface or process by which users can complete missing listing information and input it into the system.

[0916] A "prescribed platform" refers to an online service such as an auction site or flea market app where products or parts are actually listed.

[0917] "Means for recognizing emotions" refers to technologies and methods for analyzing a user's facial expressions and voice to determine their emotional state.

[0918] "Dynamic adjustment" refers to a method for changing the interface or information input process in real time according to the user's emotional state.

[0919] This invention is a system that combines an emotion engine to streamline the process and provide a user-friendly interface when a user registers or lists a product or part. Specific embodiments of this system are described below.

[0920] Hardware and software used

[0921] 1. Hardware:

[0922] Take a photo of the product using your smartphone camera or a dedicated camera.

[0923] A face detection camera is used to capture the user's facial expressions.

[0924] 2. Software:

[0925] Image analysis technology: Using OpenCV and Tesseract, product images are analyzed to identify product information.

[0926] Database Access: Uses DatabaseConnector to retrieve details from the product database based on the identified product information.

[0927] Emotion Recognition: Use the EmotionRecognizer library to recognize the user's emotional state by analyzing their facial expressions and voice.

[0928] API communication: Use APIConnector to register information on the designated listing platform.

[0929] Specific examples

[0930] The following processing procedure will explain a specific example of when a user registers a part.

[0931] Program processing

[0932] 1. Obtaining product images:

[0933] Users take a photo of the part using a smartphone or dedicated camera and upload the image to the system.

[0934] 2. Image analysis and product information identification:

[0935] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information, which is then used to identify the part.

[0936] 3. Get part information:

[0937] Based on the identified part information, the server uses the DatabaseConnector to retrieve detailed part information from an existing product database, including part number, name, characteristics, etc.

[0938] 4. View automatically generated registration information:

[0939] Based on the acquired part information, the server automatically generates registration information such as registration title, description, and initial inventory, and displays it to the user.

[0940] 5. Emotion recognition:

[0941] It uses a face detection camera to capture the user's facial expressions and analyzes their emotional state using the EmotionRecognizer library, and the analysis results are sent to the server.

[0942] 6. Dynamic Adjustment:

[0943] The server dynamically adjusts information input and the interface depending on the user's emotional state, displaying additional assistance messages and tooltips if the user feels anxious or uncertain.

[0944] 7. Completion and final confirmation:

[0945] The user enters any missing information (e.g., arrival date and part condition) and performs a final check.

[0946] 8. Listing Information:

[0947] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[0948] Prompt Sentence Examples

[0949] Below is an example of a prompt that can be used to register a part:

[0950] Prompt 1:

[0951] To register a part, take a photo of the part and upload it. Once the photo is uploaded, the system will recognize the part and automatically generate the registration information. If you have any problems or questions during the process, an assistance message will be displayed, so please follow the instructions.

[0952] response:

[0953] The part was successfully recognized. The following information was automatically generated:

[0954] Title: Part Name

[0955] Description: Part description

[0956] Initial stock: 1

[0957] Please enter any missing information (such as arrival date) and make a final confirmation.

[0958] This allows the interface to adapt to the user's emotional state, making the registration process more efficient and stress-free.

[0959] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0960] Step 1:

[0961] Users take photos of products or parts using a smartphone or dedicated camera and upload the images to the system.

[0962] Input: Product image taken by the user

[0963] Output: Product image data uploaded to the system

[0964] Specific operation: The user launches the camera app, takes a picture of the product, and then sends the image file to the server using the specified upload button.

[0965] Step 2:

[0966] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information from the product image.

[0967] Input: Uploaded product image data

[0968] Output: Text information extracted using OCR technology (e.g., part number, product name)

[0969] Specific operation: The server reads the image file, preprocesses it with OpenCV, and extracts text using Tesseract.

[0970] Step 3:

[0971] The server uses the DatabaseConnector to search an existing product database based on the extracted character information and obtains detailed part information.

[0972] Input: Character information extracted using OCR technology

[0973] Output: Detailed part information (e.g. part name, features, specs) retrieved from product database

[0974] Specific operation: The server uses the text information as a search query and DatabaseConnector to retrieve data on related parts from the database.

[0975] Step 4:

[0976] Based on the part information acquired by the server, registration information such as registration title, description, and initial inventory is automatically generated and sent to the terminal.

[0977] Input: Detailed part information retrieved from the database

[0978] Output: Auto-generated registration information

[0979] Specific operation: The server analyzes the part information, automatically formats the necessary registration information (title, description, inventory, etc.) using a template, and sends it to the user's device.

[0980] Step 5:

[0981] The terminal displays the automatically generated registration information to the user and prompts the user to enter any missing information as necessary.

[0982] Input: Auto-generated registration information sent from the server

[0983] Output: Final registration information including any supplementary information entered by the user

[0984] Specific operation: The device displays the registration information on the screen and provides a format (e.g., text entry fields) in which the user can enter any missing information.

[0985] Step 6:

[0986] It uses a face detection camera to capture the user's facial expressions, analyzes their emotional state using the EmotionRecognizer library, and sends the analysis results to the server.

[0987] Input: User facial expression capture image

[0988] Output: User's emotional state data

[0989] Specific operation: The device's camera periodically captures the user's facial expressions, inputs the data into the EmotionRecognizer library, analyzes the emotional state, and sends it to the server.

[0990] Step 7:

[0991] The server dynamically adjusts information input and interface according to the user's emotional state.

[0992] Input: User's emotional state data

[0993] Output: Dynamically adjusted interface or assist message

[0994] Specific operation: The server analyzes the received emotional state data, and if the user feels anxious or uncertain, it generates additional assistance messages or tooltips and displays them on the device.

[0995] Step 8:

[0996] The user enters any missing information and performs a final check.

[0997] Input: Missing information (e.g., arrival date, part condition)

[0998] Output: Final verified complete registration information

[0999] Specific operation: The user enters the required information and presses the final confirmation button, which sends the information to the server.

[1000] Step 9:

[1001] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[1002] Input: Complete and final registration information

[1003] Output: Notification of completion of registration to the listing platform

[1004] What happens: The server consolidates all the information, uses the APIConnector to send the necessary data to the listing platform's API, and generates a notification to complete the registration process.

[1005] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1006] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1007] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1008] [Third embodiment]

[1009] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1010] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1011] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1012] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1013] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1014] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1015] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1016] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1017] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1018] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1019] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1020] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1021] The present invention is a system that significantly reduces the effort required to put up an item at an auction or flea market. A user photographs an item and uploads the image to the system, which then automatically identifies the item information and generates listing information. Specific embodiments of this system are described below.

[1022] Overall system overview

[1023] 1. Product image acquisition

[1024] Users take a photo of the product they want to sell using a smartphone or digital camera.

[1025] The product images taken by the terminal are uploaded to the system.

[1026] 2. Image Recognition and Product Identification

[1027] The server receives the uploaded image and identifies the product using image recognition technology.

[1028] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[1029] 3. Obtaining product information

[1030] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[1031] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[1032] 4. Automatic generation of listing information

[1033] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[1034] The terminal displays the automatically generated listing information to the user.

[1035] 5. Information Complementary Interface

[1036] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[1037] The user enters the requested information and provides final confirmation.

[1038] 6. Completing the listing process

[1039] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[1040] The server uses the listing platform's API to complete the listing process.

[1041] Specific examples

[1042] For example, if a user wants to sell a Canon EOS 80D camera:

[1043] 1. The user takes a photo with the camera on their smartphone.

[1044] 2. Upload the images taken by the device to the system server.

[1045] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[1046] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[1047] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[1048] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[1049] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[1050] 8. The server processes the listing based on the input information and generated information and lists the product on the platform.

[1051] These steps allow users to list products easily and hassle-free. The system automatically generates highly accurate listing information and smoothly connects it to the listing platform.

[1052] The processing flow will be explained below.

[1053] Step 1:

[1054] Users take a photo of the item they want to sell using their smartphone or camera.

[1055] Specific operation: A user takes a photo with a Canon EOS 80D camera.

[1056] Step 2:

[1057] The terminal uploads the photographed product image to the system server.

[1058] Specific operation: Send an image file to the server via a smartphone app.

[1059] Step 3:

[1060] The server receives the uploaded image data.

[1061] Specific operation: An image file is saved in the server's storage.

[1062] Step 4:

[1063] The server analyzes the product images using image recognition technology.

[1064] Specific operation: Runs an OCR algorithm to extract text information such as "Canon EOS 80D" from the image.

[1065] Step 5:

[1066] Based on the analysis results, the server searches an existing product database to obtain product information.

[1067] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[1068] Step 6:

[1069] The server automatically generates listing information based on the product information acquired.

[1070] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[1071] Step 7:

[1072] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[1073] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[1074] Step 8:

[1075] The user enters any necessary information and performs a final check.

[1076] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[1077] Step 9:

[1078] The terminal transmits the inputted completion information to the server.

[1079] Specific operation: Data including information entered by the user is sent to the server.

[1080] Step 10:

[1081] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[1082] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[1083] Step 11:

[1084] The server notifies the user that the listing procedure is complete.

[1085] Specific operation: Notify the user of completion via email or in-app notification.

[1086] Example 1

[1087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1088] The process of listing items at conventional auctions and flea markets involves a lot of manual work, requiring time and effort. In particular, the process of taking product images, uploading the images, identifying product information, completing the information, and completing the listing procedure is complicated and places a heavy burden on users. The present invention aims to solve these problems and provide a system that allows users to easily list items.

[1089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1090] In this invention, the server includes means for acquiring product images, means for uploading the acquired images to the server, means for analyzing the uploaded images and identifying product information using image recognition technology and optical character recognition technology (OCR), means for searching an existing product database based on the identified product information to acquire product information, means for automatically generating listing information based on the acquired product information, means for prompting a user to input missing listing information, and means for listing the listing information on a predetermined platform. This enables users to list products easily and quickly without hassle.

[1091] "Product images" are photographs or illustrations that contain visual information showing the product.

[1092] The "acquisition means" refers to hardware or software for receiving product images from users and importing them into the system.

[1093] "Uploading means" refers to hardware or software that has the function of transmitting product images from a terminal to a server.

[1094] "Means for analyzing" refers to software that includes image recognition technology and optical character recognition (OCR) technology for analyzing uploaded product images and extracting product information.

[1095] "Product information" refers to information necessary to identify a product, and includes, for example, the product name, model number, and features.

[1096] An "existing product database" is a database that contains pre-registered product information (JAN code, ISBN code, product name, specifications, images, etc.).

[1097] "Product Information" is detailed information about a particular product retrieved from an existing product database.

[1098] "Means for automatic generation" refers to software that has the function of automatically creating initial settings for listing titles, product descriptions, and prices based on acquired product information.

[1099] "Missing listing information" is data that is necessary to list an item but is missing from the automatically generated information, such as the item's condition or accessories.

[1100] The "means for inputting" refers to hardware or software that provides an interface for users to provide the missing commodity information to the system.

[1101] "Means for listing" refers to hardware or software that has the functionality to send all listing information to a designated platform and list products.

[1102] "Platform" means an online marketplace for listing, buying and selling products.

[1103] The present invention is a system that aims to significantly reduce the effort required when putting up an item at an auction or flea market. Specific embodiments of the present invention will be described below.

[1104] 1. Obtaining and uploading product images

[1105] A user takes a photo of the product they want to sell using a smartphone or digital camera. This captures a "product image." The user then uses an application on their device to upload the product image to the system. The device uses an HTTP POST request to send the image to the server.

[1106] 2. Image analysis and product information extraction

[1107] The server analyzes the received product images. For the analysis, it uses image recognition technology such as Google Cloud Vision API. This technology is used to analyze the information in the image and identify the product. It also uses OCR technology such as Tesseract to extract text information from the image. This allows it to obtain "product information" such as the product name, model number, and features.

[1108] 3. Obtaining product information

[1109] The server searches an existing product database based on the extracted product information. The product database stores, for example, JAN codes, ISBN codes, product names, specifications, images, etc. The server searches this database using SQL queries to retrieve the relevant product information.

[1110] 4. Automatic generation of listing information

[1111] The server automatically generates listing titles, product descriptions, and initial price settings based on the acquired product information. Templates are used for generation, and include titles such as "Canon EOS 80D Digital SLR Camera Body" and descriptions such as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." This generated information is sent to the device and displayed to the user.

[1112] 5. Information Complementary Interface

[1113] The terminal provides the user with an interface for entering missing listing information (such as product condition and accessories). The user accesses this interface and enters the necessary information. For example, the user can select or enter the product condition as "almost new" and the accessories as "battery, charger."

[1114] 6. Completing the listing process

[1115] The server integrates all the information entered by the user and the automatically generated information, and performs the final process to list the product on the listing platform. This process is performed using the platform's API, such as eBay API or Mercari API. The server notifies the user that the listing was successful.

[1116] Specific examples

[1117] For example, if a user wants to sell a Canon EOS 80D camera:

[1118] 1. The user takes a photo with the camera on their smartphone.

[1119] 2. Upload the images taken by the device to the system server.

[1120] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using the Google Cloud Vision API.

[1121] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[1122] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[1123] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[1124] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[1125] 8. The server lists the product on the designated platform based on the input information and generated information.

[1126] Prompt Sentence Examples

[1127] Prompt: "Please explain the system that automatically identifies product information and generates listing information after uploading product images. Please include the names of the specific hardware and software used, as well as the type of data processing and calculations performed."

[1128] This allows users to list products easily and quickly without any hassle. The system utilizes generative AI models to automatically generate highly accurate listing information and smoothly connect it to the listing platform.

[1129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1130] Step 1: Get product images

[1131] Users take a photo of the item they want to sell using a smartphone or digital camera.

[1132] Specifically, the user opens the camera app on their smartphone, frames the product, and presses the capture button.

[1133] Input: Physical Goods

[1134] Output: Digital image file

[1135] Step 2: Upload an image

[1136] The terminal uploads the captured product images to the system.

[1137] The user selects the image they have taken on the application and presses the upload button.

[1138] The device sends an HTTP POST request to the server to upload the image.

[1139] Input: Digital image file

[1140] Output: Image data sent to the server

[1141] Step 3: Image recognition and product identification

[1142] The server receives the uploaded images and performs analysis.

[1143] The server uses the Google Cloud Vision API to perform image recognition.

[1144] The server uses OCR technology (e.g., Tesseract) to extract the text in the image.

[1145] Input: Image data sent to the server

[1146] Output: Product information data such as product name, model number, and features

[1147] Step 4: Obtain product information

[1148] The server searches an existing product database based on the acquired product information.

[1149] The server generates an SQL query to search the product database.

[1150] Obtain relevant product information (e.g., JAN code, product specifications, product name, etc.).

[1151] Input: Product information data

[1152] Output: Detailed product information (JAN code, product specifications, product name, etc.)

[1153] Step 5: Auto-generate listings

[1154] Based on the acquired product information, the server automatically generates initial settings for the listing title, product description, and price.

[1155] Use templates to combine information and generate it automatically.

[1156] The generated listing information is sent to the terminal and displayed to the user.

[1157] Input: Product details

[1158] Output: Listing title, product description, price information

[1159] Step 6: Information Completion Interface

[1160] The terminal provides an interface for the user to input missing listing information (e.g., product condition, accessories).

[1161] Displays text boxes and drop-down menus for users to enter or select information.

[1162] Input: Information entered by the user

[1163] Output: Added listing information (item condition, accessories, etc.)

[1164] Step 7: Complete your listing

[1165] The server generates the final listing information based on all the entered information and automatically lists it on the listing platform.

[1166] Use APIs (e.g., eBay API, Mercari API) to send listing information to the platform.

[1167] If the listing is successful, the server sends a notification to the user.

[1168] Input: Final listing information

[1169] Output: Listing completion notification to the listing platform

[1170] In this way, the system supports users in putting items up for sale efficiently and quickly, significantly reducing the amount of work required by users.

[1171] (Application example 1)

[1172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1173] Listing items on traditional auctions and flea markets requires a lot of time and effort. In particular, the process of uploading product images and manually entering the necessary information is cumbersome and burdensome for users. Setting an appropriate price for an item also requires knowledge and experience, making it difficult for beginners. Furthermore, linking listing information is often inconvenient, making it difficult to achieve a smooth listing process.

[1174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1175] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images to identify product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, means for proposing a price based on conditions entered by the user, and means for automatically linking the generated listing information to a predetermined e-commerce platform. This allows users to easily list products and automatically proposes an appropriate price, making it possible for even beginners to efficiently complete the listing procedure.

[1176] "Product images" refer to image data of products that users wish to sell, taken using a digital camera or smartphone.

[1177] "Means for analyzing images and identifying product information" refers to technology that uses image recognition technology to automatically extract and identify information such as product names and features from acquired product images.

[1178] The "means for automatically generating listing information" is a technology that automatically generates listing titles, descriptions, prices, etc. based on identified product information.

[1179] The "means for allowing the user to input missing listing information" is a technique for providing an interface for the user to manually complete missing listing information when the automatically generated listing information is missing.

[1180] The "means for listing the listing information on a predetermined platform" refers to a technology for automatically uploading the created listing information to an e-commerce platform and completing the listing procedure.

[1181] The "price suggestion method" is a technology that automatically calculates an appropriate selling price for a product based on the product's condition and accessory information entered by the user.

[1182] An "e-commerce platform" is an online marketplace for buying and selling goods.

[1183] The present invention relates to a smartphone application that automatically identifies product information and generates listing information by taking a photo of a product and uploading the image. A specific embodiment of this system will be described below.

[1184] Image Acquisition and Upload:

[1185] Users take pictures of the items they want to sell using their smartphone camera, and the images are uploaded to the server via the application.

[1186] Product Identification:

[1187] The server receives the uploaded image data and identifies product information using image recognition technology, using libraries such as Pillow (PIL) and pytesseract. The server extracts product names, model numbers, features, etc. from the product images.

[1188] Get product information:

[1189] Based on the identified product information, the server uses an external database API to retrieve detailed product information, including JAN codes, product names, specifications, and additional images.

[1190] Auto-generated listings:

[1191] The server automatically generates listing information based on the acquired product information and the product condition and accessory information entered by the user. The generated listing information includes the listing title, description, and initial price setting. To propose a price, the server calculates an appropriate price based on the information entered by the user.

[1192] Listing information integration:

[1193] The final listing information is automatically uploaded from the server via the API of the e-commerce platform, allowing users to list their products without going through complicated procedures.

[1194] Hardware and software used:

[1195] Smartphone: A device that takes product images and runs applications.

[1196] Server: A central server for image analysis and data processing.

[1197] Pillow (PIL): An image processing library.

[1198] pytesseract: An OCR (character recognition) library.

[1199] requests: A library for making HTTP requests.

[1200] External image recognition API: An API that analyzes product images and retrieves product information.

[1201] External Database API: API for retrieving product information.

[1202] Examples:

[1203] For example, if a user wants to sell a particular camera, they might use the following prompt:

[1204] Example prompt sentence:

[1205] Please analyze the product images below and identify the product name, features, condition, and accessories.

[1206] Then, generate the listing title, description, and price, and create a JSON file to upload to the listing platform.

[1207] Product image path: "path / to / product_image.jpg"

[1208] User input information:

[1209] Condition: "Like New"

[1210] Accessories: "Battery, charger"

[1211] Output format:

[1212] {

[1213] "title": "<listing title>",

[1214] "description": "<description>",

[1215] "price": <price>

[1216] }

[1217] By inputting this prompt sentence into the generative AI model, the entire process from identifying product information to automatically generating listing information can be carried out efficiently.

[1218] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1219] Step 1:

[1220] The user takes a picture of the product using the smartphone camera.

[1221] Input: Product image

[1222] Output: Photographed product image data

[1223] Specific operation: The user takes a picture of the item they want to sell and saves it on their smartphone.

[1224] Step 2:

[1225] The product images taken by the terminal are uploaded to the server.

[1226] Input: Photographed product image data

[1227] Output: Product image data uploaded to the server

[1228] Specific operation: An application on the smartphone sends the captured image file to the server.

[1229] Step 3:

[1230] The server analyzes the uploaded image data and identifies the product information.

[1231] Input: Product image data uploaded to the server

[1232] Output: Identified product information

[1233] How it works: The server uses Pillow and pytesseract to analyze the image and identify the product name and model number. For example, it uses image recognition technology to extract text information about the product.

[1234] Step 4:

[1235] The server acquires product information from an external database based on the identified product information.

[1236] Input: Identified product information

[1237] Output: Product information (e.g. JAN code, product name, specifications, additional images)

[1238] Specific operation: The server uses the recognized product name and model number as a key to call an external database API and obtain detailed product information.

[1239] Step 5:

[1240] Listing information is automatically generated based on the product information acquired by the server.

[1241] Input: Product information and user-entered information (product condition and accessories)

[1242] Output: Auto-generated listing information (title, description, price)

[1243] Specific operation: The server combines the acquired product information with the user's input information to generate an listing title, description, and initial price. For example, the description may reflect the product's unique characteristics and the product's condition input by the user.

[1244] Step 6:

[1245] The server uploads the listing information to a designated e-commerce platform.

[1246] Input: Auto-generated listing information

[1247] Output: Listing information uploaded to e-commerce platform

[1248] Specific operation: The server automatically sends the generated listing information to the API of the e-commerce platform to complete the listing process, for example, by converting the listing information into JSON format and sending it to the API.

[1249] By following the above steps, users can easily list items for sale and efficiently complete the listing procedure.

[1250] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1251] The present invention is a system that makes the listing process more efficient and user-friendly by combining an emotion engine that recognizes and analyzes the emotions of users when they list items at auctions or flea markets. Specific embodiments of this system are described below.

[1252] Overall system overview

[1253] 1. Product image acquisition

[1254] Users take a photo of the product they want to sell using a smartphone or digital camera.

[1255] The product images taken by the terminal are uploaded to the system.

[1256] 2. Image Recognition and Product Identification

[1257] The server receives the uploaded image and identifies the product using image recognition technology.

[1258] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[1259] 3. Obtaining product information

[1260] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[1261] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[1262] 4. Automatic generation of listing information

[1263] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[1264] The terminal displays the automatically generated listing information to the user.

[1265] 5. Emotion recognition

[1266] The device uses an emotion engine to analyze the user's facial expressions, voice, etc., and recognizes the user's emotional state.

[1267] The server receives the emotion recognition results and dynamically adjusts the listing information and user interface if the user is dissatisfied or has questions.

[1268] 6. Information Complementary Interface

[1269] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[1270] The user enters the requested information and provides final confirmation.

[1271] 7. Completing the listing process

[1272] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[1273] The server uses the listing platform's API to complete the listing process.

[1274] Specific examples

[1275] For example, if a user wants to sell a Canon EOS 80D camera:

[1276] 1. The user takes a photo with the camera on their smartphone.

[1277] 2. Upload the images taken by the device to the system server.

[1278] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[1279] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[1280] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[1281] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[1282] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[1283] 8. The device analyzes the user's facial expressions and voice, and if the user feels anxious or has questions, displays additional assistance messages or tooltips.

[1284] 9. The server processes the listing based on the input information and generated information and lists the product on the platform.

[1285] In this way, by combining emotion engines, interactions can be provided that correspond to the user's emotional state, improving the overall user experience of the listing process.

[1286] The processing flow will be explained below.

[1287] Step 1:

[1288] Users take a photo of the item they want to sell using their smartphone or camera.

[1289] Specific action: A user takes a photo with a Canon EOS 80D camera.

[1290] Step 2:

[1291] The terminal uploads the photographed product image to the system server.

[1292] Specific operation: Send an image file to the server via a smartphone app.

[1293] Step 3:

[1294] The server receives the uploaded image data.

[1295] Specific operation: An image file is saved in the server's storage.

[1296] Step 4:

[1297] The server analyzes the product images using image recognition technology.

[1298] Specific operation: Run the OCR algorithm and extract the text information "Canon EOS 80D" from the image.

[1299] Step 5:

[1300] Based on the analysis results, the server searches an existing product database to obtain product information.

[1301] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[1302] Step 6:

[1303] The server automatically generates listing information based on the product information acquired.

[1304] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[1305] Step 7:

[1306] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[1307] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[1308] Step 8:

[1309] The device uses an emotion engine that analyzes the user's facial expressions and voice and recognizes the user's emotions.

[1310] Specific operation: The device's camera and microphone are used to analyze the user's facial expressions and tone of voice to assess their emotional state.

[1311] Step 9:

[1312] The server receives the emotion recognition results and adjusts the interface and feedback content according to the user's emotions.

[1313] Specific action: If the user feels anxious, display a more detailed guide or assistance message.

[1314] Step 10:

[1315] The user enters any necessary information and performs a final check.

[1316] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[1317] Step 11:

[1318] The terminal transmits the inputted completion information to the server.

[1319] Specific operation: Data including information entered by the user is sent to the server.

[1320] Step 12:

[1321] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[1322] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[1323] Step 13:

[1324] The server notifies the user that the listing procedure is complete.

[1325] Specific operation: Notify the user of completion via email or in-app notification.

[1326] Example 2

[1327] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1328] The process of listing items at traditional auctions and flea markets is often time-consuming and complicated for users. In particular, the time required to enter listing information and confirm detailed product information degrades the user experience. Furthermore, a one-sided interface that ignores the user's emotional state can cause anxiety and uncertainty. This creates a challenge for users, as it hinders the overall listing process.

[1329] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring product images using an imaging device, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for recognizing the user's emotional state using an emotion engine and dynamically adjusting the interface, means for prompting the user to input missing listing information, and means for listing the generated listing information on a predetermined network platform. This enables the user to list products easily and efficiently, and a more comfortable user experience can be provided by adjusting the interface according to the user's emotional state.

[1330] A "photography device" is a device that a user uses to take product images, such as a smartphone camera or a digital camera.

[1331] The "means for acquiring product images" refers to a means by which a user takes a product image using a photographing device and imports the image into the system.

[1332] "Means for analyzing images and identifying product information" refers to means for analyzing product images acquired by the system and extracting product information (e.g., product name, model number, etc.) using image recognition technology or optical character recognition (OCR).

[1333] "Means for automatically generating listing information" refers to means by which the system automatically generates listing information such as listing title, product description, and price based on the identified product information.

[1334] An "emotion engine" refers to an algorithm or software that analyzes a user's facial expressions and voice to recognize their emotional state.

[1335] "Means for recognizing emotional states and dynamically adjusting the interface" refers to a means for using an emotion engine to know the user's emotional state and adjusting the user interface in real time according to the results.

[1336] The "means for allowing the user to input missing information for auction" refers to a means for providing an interface for allowing the user to input information when information necessary for auction is missing.

[1337] "Means for listing the generated listing information on a specified network platform" refers to means for listing a product on a network platform (e.g., an auction site or a flea market app) using the automatically generated listing information.

[1338] This invention is a system that combines an emotion engine to make the listing process efficient and user-friendly when users list items at auctions or flea markets. The system is composed of a user, a terminal, and a server.

[1339] First, a user uses a camera such as a smartphone or digital camera to take a picture of the product they want to sell. The device then uploads the captured product image to the system. The server then receives the image and identifies the product information using image analysis and optical character recognition (OCR) technology. Specifically, the server uses an image analysis library (e.g., OpenCV) and OCR technology to extract features from the product image and read text information.

[1340] Next, the server searches an existing product database based on the acquired product information to obtain detailed product information (e.g., JAN code, product specifications, etc.), issues a database query, and temporarily stores the relevant product information.

[1341] Based on this information, the server automatically generates listing information, including an automatically generated listing title, product description, and default price, using a natural language processing model (e.g., GPT-3). The generated information is sent to the device and displayed to the user.

[1342] The emotion engine then analyzes the user's emotional state. The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The server receives the emotion analysis results and dynamically adjusts the user interface if the user is dissatisfied or has questions. For example, if the user has questions, it displays additional explanations or help messages.

[1343] For any missing listing information, the terminal provides an interface for the user to input the information. Specifically, an input form for the product's condition and accessories is displayed, and the user enters the information and makes a final confirmation. The input information is confirmed by pressing the final confirmation button.

[1344] Finally, the server generates the final listing information based on all the entered information and automatically lists the item on the designated listing platform. It sends the data to the listing platform using an API to complete the listing process. It receives a response from the listing platform and notifies the user that the listing has been completed.

[1345] As a concrete example, consider a user who wants to sell a Canon EOS 80D camera. The user first takes a photo of the camera with their smartphone and uploads it to the server. The server analyzes the image and identifies it as a "Canon EOS 80D" using OCR technology. The server then searches an existing product database to retrieve accurate product information. The server then automatically generates a listing title such as "Canon EOS 80D Digital SLR Camera Body" and a description such as "High-Performance Digital SLR Camera," which the device displays to the user. The user then enters additional information, and the emotion engine adjusts the interface to address the user's concerns and questions. Finally, the server sends all listing information to the network platform, completing the listing.

[1346] Example prompt sentence:

[1347] Please explain the process of analyzing photos taken by a user with a "digital camera," displaying an automatically generated listing title and description based on detailed product information, analyzing the user's facial expressions and voice, providing assistance messages according to their emotions, and finally listing the product on a designated listing platform.

[1348] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1349] Step 1: Get product images

[1350] The user takes a photo of the product they want to sell using a photographic device (e.g., a smartphone or digital camera).

[1351] Input: Product image

[1352] The terminal will upload the captured product image to the system. At this time, the terminal will display the message "Uploading image."

[1353] Output: Product images uploaded to the system

[1354] Step 2: Image recognition and product identification

[1355] The server receives the uploaded product images.

[1356] Input: Uploaded product image

[1357] The server uses an image analysis library (e.g., OpenCV) and optical character recognition (OCR) technology to analyze product images and extract product information (e.g., product name, model number, etc.).

[1358] Data processing: Extract features from images and recognize text information.

[1359] Output: Identified product information (e.g. Canon EOS 80D)

[1360] Step 3: Obtain product information

[1361] The server searches an existing product database based on the identified product information.

[1362] Input: Identified product information

[1363] The server issues a product database query to retrieve relevant product information (e.g., JAN code, product specifications, etc.).

[1364] Data Calculation: Information retrieval and results retrieval from product databases.

[1365] Output: Detailed product information (e.g., JAN code, product specifications)

[1366] Step 4: Auto-generate listings

[1367] The server automatically generates initial listing titles, product descriptions, and prices based on the acquired product information.

[1368] Input: Detailed product information

[1369] The server generates listing information using a natural language processing model (e.g., GPT-3).

[1370] Data processing: Listing information is created using a natural language generation model based on product information.

[1371] Output: Auto-generated listing title, description, and pricing

[1372] Step 5: Emotion Recognition

[1373] The device analyzes the user's facial expressions and voice using an emotion engine.

[1374] Input: User's facial expression data, voice data

[1375] The device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time.

[1376] Data processing: Identifying emotional states using facial expression and voice analysis algorithms.

[1377] Output: User's emotional state (e.g., anxiety, doubt)

[1378] Step 6: Dynamically Adjusting the Interface

[1379] The server dynamically adjusts the user interface based on the results of the emotion recognition.

[1380] Input: User's emotional state

[1381] The device will change its interface based on the emotion engine's analysis results, and will display additional explanations and help messages as needed.

[1382] Data manipulation: Executes logic to dynamically change interface elements based on the user's emotional state.

[1383] Output: A dynamically adjusted user interface

[1384] Step 7: Information Completion Interface

[1385] The terminal provides an interface for the user to input missing listing information.

[1386] Enter: Missing listing information items

[1387] The terminal displays a form where you can enter information about the product's condition and accessories.

[1388] Data Entry: The user enters the required information into a form and submits it to the system.

[1389] Output: Additional listing information entered

[1390] Step 8: Complete your listing

[1391] The server generates the final listing information based on all the information entered.

[1392] Input: Generated listing, additional information entered by the user

[1393] The server uses the listing platform's API to complete the listing process.

[1394] Data processing: Generate comprehensive listing information and complete listing via the network platform API.

[1395] Output: Listing completion notification, response from the listing platform

[1396] (Application example 2)

[1397] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1398] Conventionally, product and part registration and management processes have often been performed manually, resulting in inefficiency and stress for users. Furthermore, because the system does not take user feelings into consideration, it is prone to operational errors and registration errors. The present invention aims to solve these problems and provide an efficient and user-friendly registration system.

[1399] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1400] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, and means for recognizing a user's emotions and dynamically adjusting information input and an interface according to the emotions. This makes it possible to identify product information using image recognition technology and adjust the interface according to the user's emotions.

[1401] "Product image" is image data of the product or part that the user wishes to register or list.

[1402] "Means for acquiring" refers to a method or device that allows a user to take product images using a camera or smartphone and upload them to the system.

[1403] The "analysis means" refers to the technology or method used to analyze the acquired product image and identify product information and part information.

[1404] "Product information" is detailed data including the model number, name, and features of a product or part.

[1405] "Listing Information" means information displayed on the listing platform, such as the listing title, description, and price.

[1406] The "means for automatic generation" refers to a technique or method by which the system automatically creates listing information based on the identified product information.

[1407] "Means for input" refers to the interface or process by which users can complete missing listing information and input it into the system.

[1408] A "prescribed platform" refers to an online service such as an auction site or flea market app where products or parts are actually listed.

[1409] "Means for recognizing emotions" refers to technologies and methods for analyzing a user's facial expressions and voice to determine their emotional state.

[1410] "Dynamic adjustment" refers to a method for changing the interface or information input process in real time according to the user's emotional state.

[1411] This invention is a system that combines an emotion engine to streamline the process and provide a user-friendly interface when a user registers or lists a product or part. Specific embodiments of this system are described below.

[1412] Hardware and software used

[1413] 1. Hardware:

[1414] Take a photo of the product using your smartphone camera or a dedicated camera.

[1415] A face detection camera is used to capture the user's facial expressions.

[1416] 2. Software:

[1417] Image analysis technology: Using OpenCV and Tesseract, product images are analyzed to identify product information.

[1418] Database Access: Uses DatabaseConnector to retrieve details from the product database based on the identified product information.

[1419] Emotion Recognition: Use the EmotionRecognizer library to recognize the user's emotional state by analyzing their facial expressions and voice.

[1420] API communication: Use APIConnector to register information on the designated listing platform.

[1421] Specific examples

[1422] The following processing procedure will explain a specific example of when a user registers a part.

[1423] Program processing

[1424] 1. Obtaining product images:

[1425] Users take a photo of the part using a smartphone or dedicated camera and upload the image to the system.

[1426] 2. Image analysis and product information identification:

[1427] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information, which is then used to identify the part.

[1428] 3. Get part information:

[1429] Based on the identified part information, the server uses the DatabaseConnector to retrieve detailed part information from an existing product database, including part number, name, characteristics, etc.

[1430] 4. View automatically generated registration information:

[1431] Based on the acquired part information, the server automatically generates registration information such as registration title, description, and initial inventory, and displays it to the user.

[1432] 5. Emotion recognition:

[1433] It uses a face detection camera to capture the user's facial expressions and analyzes their emotional state using the EmotionRecognizer library, and the analysis results are sent to the server.

[1434] 6. Dynamic Adjustment:

[1435] The server dynamically adjusts information input and the interface depending on the user's emotional state, displaying additional assistance messages and tooltips if the user feels anxious or uncertain.

[1436] 7. Completion and final confirmation:

[1437] The user enters any missing information (e.g., arrival date and part condition) and performs a final check.

[1438] 8. Listing Information:

[1439] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[1440] Prompt Sentence Examples

[1441] Below is an example of a prompt that can be used to register a part:

[1442] Prompt 1:

[1443] To register a part, take a photo of the part and upload it. Once the photo is uploaded, the system will recognize the part and automatically generate the registration information. If you have any problems or questions during the process, an assistance message will be displayed, so please follow the instructions.

[1444] response:

[1445] The part was successfully recognized. The following information was automatically generated:

[1446] Title: Part Name

[1447] Description: Part description

[1448] Initial stock: 1

[1449] Please enter any missing information (such as arrival date) and make a final confirmation.

[1450] This allows the interface to adapt to the user's emotional state, making the registration process more efficient and stress-free.

[1451] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1452] Step 1:

[1453] Users take photos of products or parts using a smartphone or dedicated camera and upload the images to the system.

[1454] Input: Product image taken by the user

[1455] Output: Product image data uploaded to the system

[1456] Specific operation: The user launches the camera app, takes a picture of the product, and then sends the image file to the server using the specified upload button.

[1457] Step 2:

[1458] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information from the product image.

[1459] Input: Uploaded product image data

[1460] Output: Text information extracted using OCR technology (e.g., part number, product name)

[1461] Specific operation: The server reads the image file, preprocesses it with OpenCV, and extracts text using Tesseract.

[1462] Step 3:

[1463] Based on the extracted character information, the server uses DatabaseConnector to search an existing product database and obtain detailed part information.

[1464] Input: Character information extracted using OCR technology

[1465] Output: Detailed part information (e.g. part name, features, specs) retrieved from product database

[1466] Specific operation: The server uses the text information as a search query and DatabaseConnector to retrieve data on related parts from the database.

[1467] Step 4:

[1468] Based on the part information acquired by the server, registration information such as registration title, description, and initial inventory is automatically generated and sent to the terminal.

[1469] Input: Detailed part information retrieved from the database

[1470] Output: Auto-generated registration information

[1471] Specific operation: The server analyzes the part information, automatically formats the necessary registration information (title, description, inventory, etc.) using a template, and sends it to the user's device.

[1472] Step 5:

[1473] The terminal displays the automatically generated registration information to the user and prompts the user to enter any missing information as necessary.

[1474] Input: Auto-generated registration information sent from the server

[1475] Output: Final registration information including any supplementary information entered by the user

[1476] Specific operation: The device displays the registration information on the screen and provides a format (e.g., text entry fields) in which the user can enter any missing information.

[1477] Step 6:

[1478] It uses a face detection camera to capture the user's facial expressions, analyzes their emotional state using the EmotionRecognizer library, and sends the analysis results to the server.

[1479] Input: User facial expression capture image

[1480] Output: User's emotional state data

[1481] Specific operation: The device's camera periodically captures the user's facial expressions, inputs the data into the EmotionRecognizer library, analyzes the emotional state, and sends it to the server.

[1482] Step 7:

[1483] The server dynamically adjusts information input and interface according to the user's emotional state.

[1484] Input: User's emotional state data

[1485] Output: Dynamically adjusted interface or assist message

[1486] Specific operation: The server analyzes the received emotional state data, and if the user feels anxious or uncertain, it generates additional assistance messages or tooltips and displays them on the device.

[1487] Step 8:

[1488] The user enters any missing information and performs a final check.

[1489] Input: Missing information (e.g., arrival date, part condition)

[1490] Output: Final verified complete registration information

[1491] Specific operation: The user enters the required information and presses the final confirmation button, which sends the information to the server.

[1492] Step 9:

[1493] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[1494] Input: Complete and final registration information

[1495] Output: Notification of completion of registration to the listing platform

[1496] What happens: The server consolidates all the information, uses the APIConnector to send the necessary data to the listing platform's API, and generates a notification to complete the registration process.

[1497] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1498] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1499] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1500] [Fourth embodiment]

[1501] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1502] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1503] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1504] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1505] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1506] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1507] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1508] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1509] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1510] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1511] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1512] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1513] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1514] The present invention is a system that significantly reduces the effort required to put up an item at an auction or flea market. A user photographs an item and uploads the image to the system, which then automatically identifies the item information and generates listing information. Specific embodiments of this system are described below.

[1515] Overall system overview

[1516] 1. Product image acquisition

[1517] Users take a photo of the product they want to sell using a smartphone or digital camera.

[1518] The product images taken by the terminal are uploaded to the system.

[1519] 2. Image Recognition and Product Identification

[1520] The server receives the uploaded image and identifies the product using image recognition technology.

[1521] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[1522] 3. Obtaining product information

[1523] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[1524] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[1525] 4. Automatic generation of listing information

[1526] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[1527] The terminal displays the automatically generated listing information to the user.

[1528] 5. Information Complementary Interface

[1529] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[1530] The user enters the requested information and provides final confirmation.

[1531] 6. Completing the listing process

[1532] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[1533] The server uses the listing platform's API to complete the listing process.

[1534] Specific examples

[1535] For example, if a user wants to sell a Canon EOS 80D camera:

[1536] 1. The user takes a photo with the camera on their smartphone.

[1537] 2. Upload the images taken by the device to the system server.

[1538] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[1539] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[1540] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[1541] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[1542] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[1543] 8. The server processes the listing based on the input information and generated information and lists the product on the platform.

[1544] These steps allow users to list products easily and hassle-free. The system automatically generates highly accurate listing information and smoothly connects it to the listing platform.

[1545] The processing flow will be explained below.

[1546] Step 1:

[1547] Users take a photo of the item they want to sell using their smartphone or camera.

[1548] Specific operation: A user takes a photo with a Canon EOS 80D camera.

[1549] Step 2:

[1550] The terminal uploads the photographed product image to the system server.

[1551] Specific operation: Send an image file to the server via a smartphone app.

[1552] Step 3:

[1553] The server receives the uploaded image data.

[1554] Specific operation: An image file is saved in the server's storage.

[1555] Step 4:

[1556] The server analyzes the product images using image recognition technology.

[1557] Specific operation: Runs an OCR algorithm to extract text information such as "Canon EOS 80D" from the image.

[1558] Step 5:

[1559] Based on the analysis results, the server searches an existing product database to obtain product information.

[1560] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[1561] Step 6:

[1562] The server automatically generates listing information based on the product information acquired.

[1563] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[1564] Step 7:

[1565] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[1566] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[1567] Step 8:

[1568] The user enters any necessary information and performs a final check.

[1569] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[1570] Step 9:

[1571] The terminal transmits the inputted completion information to the server.

[1572] Specific operation: Data including information entered by the user is sent to the server.

[1573] Step 10:

[1574] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[1575] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[1576] Step 11:

[1577] The server notifies the user that the listing procedure is complete.

[1578] Specific operation: Notify the user of completion via email or in-app notification.

[1579] Example 1

[1580] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1581] The process of listing items at conventional auctions and flea markets involves a lot of manual work, requiring time and effort. In particular, the process of taking product images, uploading the images, identifying product information, completing the information, and completing the listing procedure is complicated and places a heavy burden on users. The present invention aims to solve these problems and provide a system that allows users to easily list items.

[1582] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1583] In this invention, the server includes means for acquiring product images, means for uploading the acquired images to the server, means for analyzing the uploaded images and identifying product information using image recognition technology and optical character recognition technology (OCR), means for searching an existing product database based on the identified product information to acquire product information, means for automatically generating listing information based on the acquired product information, means for prompting a user to input missing listing information, and means for listing the listing information on a predetermined platform. This enables users to list products easily and quickly without hassle.

[1584] "Product images" are photographs or illustrations that contain visual information showing the product.

[1585] The "acquisition means" refers to hardware or software for receiving product images from users and importing them into the system.

[1586] "Uploading means" refers to hardware or software that has the function of transmitting product images from a terminal to a server.

[1587] "Means for analyzing" refers to software that includes image recognition technology and optical character recognition (OCR) technology for analyzing uploaded product images and extracting product information.

[1588] "Product information" refers to information necessary to identify a product, and includes, for example, the product name, model number, and features.

[1589] An "existing product database" is a database that contains pre-registered product information (JAN code, ISBN code, product name, specifications, images, etc.).

[1590] "Product Information" is detailed information about a particular product retrieved from an existing product database.

[1591] "Means for automatic generation" refers to software that has the function of automatically creating initial settings for listing titles, product descriptions, and prices based on acquired product information.

[1592] "Missing listing information" is data that is necessary to list an item but is missing from the automatically generated information, such as the item's condition or accessories.

[1593] The "means for inputting" refers to hardware or software that provides an interface for users to provide the missing commodity information to the system.

[1594] "Means for listing" refers to hardware or software that has the functionality to send all listing information to a designated platform and list products.

[1595] "Platform" means an online marketplace for listing, buying and selling products.

[1596] The present invention is a system that aims to significantly reduce the effort required when putting up an item at an auction or flea market. Specific embodiments of the present invention will be described below.

[1597] 1. Obtaining and uploading product images

[1598] A user takes a photo of the product they want to sell using a smartphone or digital camera. This captures a "product image." The user then uses an application on their device to upload the product image to the system. The device uses an HTTP POST request to send the image to the server.

[1599] 2. Image analysis and product information extraction

[1600] The server analyzes the received product images. For the analysis, it uses image recognition technology such as Google Cloud Vision API. This technology is used to analyze the information in the image and identify the product. It also uses OCR technology such as Tesseract to extract text information from the image. This allows it to obtain "product information" such as the product name, model number, and features.

[1601] 3. Obtaining product information

[1602] The server searches an existing product database based on the extracted product information. The product database stores, for example, JAN codes, ISBN codes, product names, specifications, images, etc. The server searches this database using SQL queries to retrieve the relevant product information.

[1603] 4. Automatic generation of listing information

[1604] The server automatically generates listing titles, product descriptions, and initial price settings based on the acquired product information. Templates are used for generation, and include titles such as "Canon EOS 80D Digital SLR Camera Body" and descriptions such as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." This generated information is sent to the device and displayed to the user.

[1605] 5. Information Complementary Interface

[1606] The terminal provides the user with an interface for entering missing listing information (such as product condition and accessories). The user accesses this interface and enters the necessary information. For example, the user can select or enter the product condition as "almost new" and the accessories as "battery, charger."

[1607] 6. Completing the listing process

[1608] The server integrates all the information entered by the user and the automatically generated information, and performs the final process to list the product on the listing platform. This process is performed using the platform's API, such as eBay API or Mercari API. The server notifies the user that the listing was successful.

[1609] Specific examples

[1610] For example, if a user wants to sell a Canon EOS 80D camera:

[1611] 1. The user takes a photo with the camera on their smartphone.

[1612] 2. Upload the images taken by the device to the system server.

[1613] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using the Google Cloud Vision API.

[1614] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[1615] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[1616] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[1617] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[1618] 8. The server lists the product on the designated platform based on the input information and generated information.

[1619] Prompt Sentence Examples

[1620] Prompt: "Please explain the system that automatically identifies product information and generates listing information after uploading product images. Please include the names of the specific hardware and software used, as well as the type of data processing and calculations performed."

[1621] This allows users to list products easily and quickly without any hassle. The system utilizes generative AI models to automatically generate highly accurate listing information and smoothly connect it to the listing platform.

[1622] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1623] Step 1: Get product images

[1624] Users take a photo of the item they want to sell using a smartphone or digital camera.

[1625] Specifically, the user opens the camera app on their smartphone, frames the product, and presses the capture button.

[1626] Input: Physical Goods

[1627] Output: Digital image file

[1628] Step 2: Upload an image

[1629] The terminal uploads the captured product images to the system.

[1630] The user selects the image they have taken on the application and presses the upload button.

[1631] The device sends an HTTP POST request to the server to upload the image.

[1632] Input: Digital image file

[1633] Output: Image data sent to the server

[1634] Step 3: Image recognition and product identification

[1635] The server receives the uploaded images and performs analysis.

[1636] The server uses the Google Cloud Vision API to perform image recognition.

[1637] The server uses OCR technology (e.g., Tesseract) to extract the text in the image.

[1638] Input: Image data sent to the server

[1639] Output: Product information data such as product name, model number, and features

[1640] Step 4: Obtain product information

[1641] The server searches an existing product database based on the acquired product information.

[1642] The server generates an SQL query to search the product database.

[1643] Obtain relevant product information (e.g., JAN code, product specifications, product name, etc.).

[1644] Input: Product information data

[1645] Output: Detailed product information (JAN code, product specifications, product name, etc.)

[1646] Step 5: Auto-generate listings

[1647] Based on the acquired product information, the server automatically generates initial settings for the listing title, product description, and price.

[1648] Use templates to combine information and generate it automatically.

[1649] The generated listing information is sent to the terminal and displayed to the user.

[1650] Input: Product details

[1651] Output: Listing title, product description, price information

[1652] Step 6: Information Completion Interface

[1653] The terminal provides an interface for the user to input missing listing information (e.g., product condition, accessories).

[1654] Displays text boxes and drop-down menus for users to enter or select information.

[1655] Input: Information entered by the user

[1656] Output: Added listing information (item condition, accessories, etc.)

[1657] Step 7: Complete your listing

[1658] The server generates the final listing information based on all the entered information and automatically lists it on the listing platform.

[1659] Use APIs (e.g., eBay API, Mercari API) to send listing information to the platform.

[1660] If the listing is successful, the server sends a notification to the user.

[1661] Input: Final listing information

[1662] Output: Listing completion notification to the listing platform

[1663] In this way, the system supports users in putting items up for sale efficiently and quickly, significantly reducing the amount of work required by users.

[1664] (Application example 1)

[1665] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1666] Listing items on traditional auctions and flea markets requires a lot of time and effort. In particular, the process of uploading product images and manually entering the necessary information is cumbersome and burdensome for users. Setting an appropriate price for an item also requires knowledge and experience, making it difficult for beginners. Furthermore, linking listing information is often inconvenient, making it difficult to achieve a smooth listing process.

[1667] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1668] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images to identify product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, means for proposing a price based on conditions entered by the user, and means for automatically linking the generated listing information to a predetermined e-commerce platform. This allows users to easily list products and automatically proposes an appropriate price, making it possible for even beginners to efficiently complete the listing procedure.

[1669] "Product images" refer to image data of products that users wish to sell, taken using a digital camera or smartphone.

[1670] "Means for analyzing images and identifying product information" refers to technology that uses image recognition technology to automatically extract and identify information such as product names and features from acquired product images.

[1671] The "means for automatically generating listing information" is a technology that automatically generates listing titles, descriptions, prices, etc. based on identified product information.

[1672] The "means for allowing the user to input missing listing information" is a technique for providing an interface for the user to manually complete missing listing information when the automatically generated listing information is missing.

[1673] The "means for listing the listing information on a predetermined platform" refers to a technology for automatically uploading the created listing information to an e-commerce platform and completing the listing procedure.

[1674] The "price suggestion method" is a technology that automatically calculates an appropriate selling price for a product based on the product's condition and accessory information entered by the user.

[1675] An "e-commerce platform" is an online marketplace for buying and selling goods.

[1676] The present invention relates to a smartphone application that automatically identifies product information and generates listing information by taking a photo of a product and uploading the image. A specific embodiment of this system will be described below.

[1677] Image Acquisition and Upload:

[1678] Users take pictures of the items they want to sell using their smartphone camera, and the images are uploaded to the server via the application.

[1679] Product Identification:

[1680] The server receives the uploaded image data and identifies product information using image recognition technology, using libraries such as Pillow (PIL) and pytesseract. The server extracts product names, model numbers, features, etc. from the product images.

[1681] Get product information:

[1682] Based on the identified product information, the server uses an external database API to retrieve detailed product information, including JAN codes, product names, specifications, and additional images.

[1683] Auto-generated listings:

[1684] The server automatically generates listing information based on the acquired product information and the product condition and accessory information entered by the user. The generated listing information includes the listing title, description, and initial price setting. To propose a price, the server calculates an appropriate price based on the information entered by the user.

[1685] Listing information integration:

[1686] The final listing information is automatically uploaded from the server via the API of the e-commerce platform, allowing users to list their products without going through complicated procedures.

[1687] Hardware and software used:

[1688] Smartphone: A device that takes product images and runs applications.

[1689] Server: A central server for image analysis and data processing.

[1690] Pillow (PIL): An image processing library.

[1691] pytesseract: An OCR (character recognition) library.

[1692] requests: A library for making HTTP requests.

[1693] External image recognition API: An API that analyzes product images and retrieves product information.

[1694] External Database API: API for retrieving product information.

[1695] Examples:

[1696] For example, if a user wants to sell a particular camera, they might use the following prompt:

[1697] Example prompt sentence:

[1698] Please analyze the product images below and identify the product name, features, condition, and accessories.

[1699] Then, generate the listing title, description, and price, and create a JSON file to upload to the listing platform.

[1700] Product image path: "path / to / product_image.jpg"

[1701] User input information:

[1702] Condition: "Like New"

[1703] Accessories: "Battery, charger"

[1704] Output format:

[1705] {

[1706] "title": "<listing title>",

[1707] "description": "<description>",

[1708] "price": <price>

[1709] }

[1710] By inputting this prompt sentence into the generative AI model, the entire process from identifying product information to automatically generating listing information can be carried out efficiently.

[1711] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1712] Step 1:

[1713] The user takes a picture of the product using the smartphone camera.

[1714] Input: Product image

[1715] Output: Photographed product image data

[1716] Specific operation: The user takes a picture of the item they want to sell and saves it on their smartphone.

[1717] Step 2:

[1718] The product images taken by the terminal are uploaded to the server.

[1719] Input: Photographed product image data

[1720] Output: Product image data uploaded to the server

[1721] Specific operation: An application on the smartphone sends the captured image file to the server.

[1722] Step 3:

[1723] The server analyzes the uploaded image data and identifies the product information.

[1724] Input: Product image data uploaded to the server

[1725] Output: Identified product information

[1726] How it works: The server uses Pillow and pytesseract to analyze the image and identify the product name and model number. For example, it uses image recognition technology to extract text information about the product.

[1727] Step 4:

[1728] The server acquires product information from an external database based on the identified product information.

[1729] Input: Identified product information

[1730] Output: Product information (e.g. JAN code, product name, specifications, additional images)

[1731] Specific operation: The server uses the recognized product name and model number as a key to call an external database API and obtain detailed product information.

[1732] Step 5:

[1733] Listing information is automatically generated based on the product information acquired by the server.

[1734] Input: Product information and user-entered information (product condition and accessories)

[1735] Output: Auto-generated listing information (title, description, price)

[1736] Specific operation: The server combines the acquired product information with the user's input information to generate an listing title, description, and initial price. For example, the description may reflect the product's unique characteristics and the product's condition input by the user.

[1737] Step 6:

[1738] The server uploads the listing information to a designated e-commerce platform.

[1739] Input: Auto-generated listing information

[1740] Output: Listing information uploaded to e-commerce platform

[1741] Specific operation: The server automatically sends the generated listing information to the API of the e-commerce platform to complete the listing process, for example, by converting the listing information into JSON format and sending it to the API.

[1742] By following the above steps, users can easily list items for sale and efficiently complete the listing procedure.

[1743] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1744] The present invention is a system that makes the listing process more efficient and user-friendly by combining an emotion engine that recognizes and analyzes the emotions of users when they list items at auctions or flea markets. Specific embodiments of this system are described below.

[1745] Overall system overview

[1746] 1. Product image acquisition

[1747] Users take a photo of the product they want to sell using a smartphone or digital camera.

[1748] The product images taken by the terminal are uploaded to the system.

[1749] 2. Image Recognition and Product Identification

[1750] The server receives the uploaded image and identifies the product using image recognition technology.

[1751] The server uses OCR or other image analysis techniques to extract product information (e.g., product name, model number, features).

[1752] 3. Obtaining product information

[1753] The server searches an existing product database based on the extracted product information to obtain detailed product information.

[1754] The database contains information such as JAN codes, ISBN codes, product names, specifications, and images.

[1755] 4. Automatic generation of listing information

[1756] Based on the product information acquired by the server, the initial listing title, product description, and price are automatically generated.

[1757] The terminal displays the automatically generated listing information to the user.

[1758] 5. Emotion recognition

[1759] The device uses an emotion engine to analyze the user's facial expressions, voice, etc., and recognizes the user's emotional state.

[1760] The server receives the emotion recognition results and dynamically adjusts the listing information and user interface if the user is dissatisfied or has questions.

[1761] 6. Information Complementary Interface

[1762] The terminal provides an interface for the user to input missing information (e.g., product condition, accessories).

[1763] The user enters the requested information and provides final confirmation.

[1764] 7. Completing the listing process

[1765] The server generates final listing information based on all the input information and automatically lists the items on a predetermined listing platform.

[1766] The server uses the listing platform's API to complete the listing process.

[1767] Specific examples

[1768] For example, if a user wants to sell a Canon EOS 80D camera:

[1769] 1. The user takes a photo with the camera on their smartphone.

[1770] 2. Upload the images taken by the device to the system server.

[1771] 3. The server analyzes the received image and identifies it as a "Canon EOS 80D" using OCR technology.

[1772] 4. The server searches the existing product database and retrieves product information (e.g., JAN code, product specifications, etc.) corresponding to the Canon EOS 80D.

[1773] 5. Based on the information obtained by the server, the listing title "Canon EOS 80D Digital SLR Camera Body" and description "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality." are automatically generated.

[1774] 6. The terminal displays the generated listing information to the user and displays an interface requesting input about the product's condition and accessories.

[1775] 7. The user selects the product condition as "Like New" and the accessories as "Battery, Charger" and then makes a final check.

[1776] 8. The device analyzes the user's facial expressions and voice, and if the user feels anxious or has questions, displays additional assistance messages or tooltips.

[1777] 9. The server processes the listing based on the input information and generated information and lists the product on the platform.

[1778] In this way, by combining emotion engines, interactions can be provided that correspond to the user's emotional state, improving the overall user experience of the listing process.

[1779] The processing flow will be explained below.

[1780] Step 1:

[1781] Users take a photo of the item they want to sell using their smartphone or camera.

[1782] Specific action: A user takes a photo with a Canon EOS 80D camera.

[1783] Step 2:

[1784] The terminal uploads the photographed product image to the system server.

[1785] Specific operation: Send an image file to the server via a smartphone app.

[1786] Step 3:

[1787] The server receives the uploaded image data.

[1788] Specific operation: An image file is saved in the server's storage.

[1789] Step 4:

[1790] The server analyzes the product images using image recognition technology.

[1791] Specific operation: Run the OCR algorithm and extract the text information "Canon EOS 80D" from the image.

[1792] Step 5:

[1793] Based on the analysis results, the server searches an existing product database to obtain product information.

[1794] Specific operation: Perform a database query to obtain product information (e.g., JAN code, product name, specifications) corresponding to "Canon EOS 80D."

[1795] Step 6:

[1796] The server automatically generates listing information based on the product information acquired.

[1797] Specific operation: Generate the product title as "Canon EOS 80D Digital SLR Camera Body" and the description as "High-performance digital SLR camera, Canon EOS 80D. Equipped with a 24.2-megapixel CMOS sensor, Dual Pixel CMOS AF, and Wi-Fi functionality."

[1798] Step 7:

[1799] The terminal displays the automatically generated listing information to the user and prompts the user to enter any missing information.

[1800] Specific operation: Displays check boxes on the app UI for the product condition (new, like new, used) and accessories (battery, charger, etc.).

[1801] Step 8:

[1802] The device uses an emotion engine that analyzes the user's facial expressions and voice and recognizes the user's emotions.

[1803] Specific operation: The device's camera and microphone are used to analyze the user's facial expressions and tone of voice to assess their emotional state.

[1804] Step 9:

[1805] The server receives the emotion recognition results and adjusts the interface and feedback content according to the user's emotions.

[1806] Specific action: If the user feels anxious, display a more detailed guide or assistance message.

[1807] Step 10:

[1808] The user enters any necessary information and performs a final check.

[1809] Specific operation: Enter the product condition as "Almost new" and the accessories as "Battery, charger" and press the confirmation button.

[1810] Step 11:

[1811] The terminal transmits the inputted completion information to the server.

[1812] Specific operation: Data including information entered by the user is sent to the server.

[1813] Step 12:

[1814] The server generates the final listing information and carries out the listing procedure on a predetermined listing platform.

[1815] Specific operation: The final listing information is sent using the listing platform's API, and the product is listed.

[1816] Step 13:

[1817] The server notifies the user that the listing procedure is complete.

[1818] Specific operation: Notify the user of completion via email or in-app notification.

[1819] Example 2

[1820] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1821] The process of listing items at traditional auctions and flea markets is often time-consuming and complicated for users. In particular, the time required to enter listing information and confirm detailed product information degrades the user experience. Furthermore, a one-sided interface that ignores the user's emotional state can cause anxiety and uncertainty. This creates a challenge for users, as it hinders the overall listing process.

[1822] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring product images using an imaging device, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for recognizing the user's emotional state using an emotion engine and dynamically adjusting the interface, means for prompting the user to input missing listing information, and means for listing the generated listing information on a predetermined network platform. This enables the user to list products easily and efficiently, and a more comfortable user experience can be provided by adjusting the interface according to the user's emotional state.

[1823] A "photography device" is a device that a user uses to take product images, such as a smartphone camera or a digital camera.

[1824] The "means for acquiring product images" refers to a means by which a user takes a product image using a photographing device and imports the image into the system.

[1825] "Means for analyzing images and identifying product information" refers to means for analyzing product images acquired by the system and extracting product information (e.g., product name, model number, etc.) using image recognition technology or optical character recognition (OCR).

[1826] "Means for automatically generating listing information" refers to means by which the system automatically generates listing information such as listing title, product description, and price based on the identified product information.

[1827] An "emotion engine" refers to an algorithm or software that analyzes a user's facial expressions and voice to recognize their emotional state.

[1828] "Means for recognizing emotional states and dynamically adjusting the interface" refers to a means for using an emotion engine to know the user's emotional state and adjusting the user interface in real time according to the results.

[1829] The "means for allowing the user to input missing information for auction" refers to a means for providing an interface for allowing the user to input information when information necessary for auction is missing.

[1830] "Means for listing the generated listing information on a specified network platform" refers to means for listing a product on a network platform (e.g., an auction site or a flea market app) using the automatically generated listing information.

[1831] This invention is a system that combines an emotion engine to make the listing process efficient and user-friendly when users list items at auctions or flea markets. The system is composed of a user, a terminal, and a server.

[1832] First, a user uses a camera such as a smartphone or digital camera to take a picture of the product they want to sell. The device then uploads the captured product image to the system. The server then receives the image and identifies the product information using image analysis and optical character recognition (OCR) technology. Specifically, the server uses an image analysis library (e.g., OpenCV) and OCR technology to extract features from the product image and read text information.

[1833] Next, the server searches an existing product database based on the acquired product information to obtain detailed product information (e.g., JAN code, product specifications, etc.), issues a database query, and temporarily stores the relevant product information.

[1834] Based on this information, the server automatically generates listing information, including an automatically generated listing title, product description, and default price, using a natural language processing model (e.g., GPT-3). The generated information is sent to the device and displayed to the user.

[1835] The emotion engine then analyzes the user's emotional state. The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The server receives the emotion analysis results and dynamically adjusts the user interface if the user is dissatisfied or has questions. For example, if the user has questions, it displays additional explanations or help messages.

[1836] For any missing listing information, the terminal provides an interface for the user to input the information. Specifically, an input form for the product's condition and accessories is displayed, and the user enters the information and makes a final confirmation. The input information is confirmed by pressing the final confirmation button.

[1837] Finally, the server generates the final listing information based on all the entered information and automatically lists the item on the designated listing platform. It sends the data to the listing platform using an API to complete the listing process. It receives a response from the listing platform and notifies the user that the listing has been completed.

[1838] As a concrete example, consider a user who wants to sell a Canon EOS 80D camera. The user first takes a photo of the camera with their smartphone and uploads it to the server. The server analyzes the image and identifies it as a "Canon EOS 80D" using OCR technology. The server then searches an existing product database to retrieve accurate product information. The server then automatically generates a listing title such as "Canon EOS 80D Digital SLR Camera Body" and a description such as "High-Performance Digital SLR Camera," which the device displays to the user. The user then enters additional information, and the emotion engine adjusts the interface to address the user's concerns and questions. Finally, the server sends all listing information to the network platform, completing the listing.

[1839] Example prompt sentence:

[1840] Please explain the process of analyzing photos taken by a user with a "digital camera," displaying an automatically generated listing title and description based on detailed product information, analyzing the user's facial expressions and voice, providing assistance messages according to their emotions, and finally listing the product on a designated listing platform.

[1841] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1842] Step 1: Get product images

[1843] The user takes a photo of the product they want to sell using a photographic device (e.g., a smartphone or digital camera).

[1844] Input: Product image

[1845] The terminal will upload the captured product image to the system. At this time, the terminal will display the message "Uploading image."

[1846] Output: Product images uploaded to the system

[1847] Step 2: Image recognition and product identification

[1848] The server receives the uploaded product images.

[1849] Input: Uploaded product image

[1850] The server uses an image analysis library (e.g., OpenCV) and optical character recognition (OCR) technology to analyze product images and extract product information (e.g., product name, model number, etc.).

[1851] Data processing: Extract features from images and recognize text information.

[1852] Output: Identified product information (e.g. Canon EOS 80D)

[1853] Step 3: Obtain product information

[1854] The server searches an existing product database based on the identified product information.

[1855] Input: Identified product information

[1856] The server issues a product database query to retrieve relevant product information (e.g., JAN code, product specifications, etc.).

[1857] Data Calculation: Information retrieval and results retrieval from product databases.

[1858] Output: Detailed product information (e.g., JAN code, product specifications)

[1859] Step 4: Auto-generate listings

[1860] The server automatically generates initial listing titles, product descriptions, and prices based on the acquired product information.

[1861] Input: Detailed product information

[1862] The server generates listing information using a natural language processing model (e.g., GPT-3).

[1863] Data processing: Listing information is created using a natural language generation model based on product information.

[1864] Output: Auto-generated listing title, description, and pricing

[1865] Step 5: Emotion Recognition

[1866] The device analyzes the user's facial expressions and voice using an emotion engine.

[1867] Input: User's facial expression data, voice data

[1868] The device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time.

[1869] Data processing: Identifying emotional states using facial expression and voice analysis algorithms.

[1870] Output: User's emotional state (e.g., anxiety, doubt)

[1871] Step 6: Dynamically Adjusting the Interface

[1872] The server dynamically adjusts the user interface based on the results of the emotion recognition.

[1873] Input: User's emotional state

[1874] The device will change its interface based on the emotion engine's analysis results, and will display additional explanations and help messages as needed.

[1875] Data manipulation: Executes logic to dynamically change interface elements based on the user's emotional state.

[1876] Output: A dynamically adjusted user interface

[1877] Step 7: Information Completion Interface

[1878] The terminal provides an interface for the user to input missing listing information.

[1879] Enter: Missing listing information items

[1880] The terminal displays a form where you can enter information about the product's condition and accessories.

[1881] Data Entry: The user enters the required information into a form and submits it to the system.

[1882] Output: Additional listing information entered

[1883] Step 8: Complete your listing

[1884] The server generates the final listing information based on all the information entered.

[1885] Input: Generated listing, additional information entered by the user

[1886] The server uses the listing platform's API to complete the listing process.

[1887] Data processing: Generate comprehensive listing information and complete listing via the network platform API.

[1888] Output: Listing completion notification, response from the listing platform

[1889] (Application example 2)

[1890] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1891] Conventionally, product and part registration and management processes have often been performed manually, resulting in inefficiency and stress for users. Furthermore, because the system does not take user feelings into consideration, it is prone to operational errors and registration errors. The present invention aims to solve these problems and provide an efficient and user-friendly registration system.

[1892] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1893] In this invention, the server includes means for acquiring product images, means for analyzing the acquired images and identifying product information, means for automatically generating listing information based on the identified product information, means for prompting a user to input missing listing information, means for listing the listing information on a predetermined platform, and means for recognizing a user's emotions and dynamically adjusting information input and an interface according to the emotions. This makes it possible to identify product information using image recognition technology and adjust the interface according to the user's emotions.

[1894] "Product image" is image data of the product or part that the user wishes to register or list.

[1895] "Means of acquisition" refers to a method or device that allows a user to take product images using a camera or smartphone and upload them to the system.

[1896] The "analysis means" refers to the technology or method used to analyze the acquired product image and identify product information and part information.

[1897] "Product information" is detailed data including the model number, name, and features of a product or part.

[1898] "Listing Information" means information displayed on the listing platform, such as the listing title, description, and price.

[1899] The "means for automatic generation" refers to a technique or method by which the system automatically creates listing information based on the identified product information.

[1900] "Means for input" refers to the interface or process by which users can complete missing listing information and input it into the system.

[1901] A "prescribed platform" refers to an online service such as an auction site or flea market app where products or parts are actually listed.

[1902] "Means for recognizing emotions" refers to technologies and methods for analyzing a user's facial expressions and voice to determine their emotional state.

[1903] "Dynamic adjustment" refers to a method for changing the interface or information input process in real time according to the user's emotional state.

[1904] This invention is a system that combines an emotion engine to streamline the process and provide a user-friendly interface when a user registers or lists a product or part. Specific embodiments of this system are described below.

[1905] Hardware and software used

[1906] 1. Hardware:

[1907] Take a photo of the product using your smartphone camera or a dedicated camera.

[1908] A face detection camera is used to capture the user's facial expressions.

[1909] 2. Software:

[1910] Image analysis technology: Using OpenCV and Tesseract, product images are analyzed to identify product information.

[1911] Database Access: Uses DatabaseConnector to retrieve details from the product database based on the identified product information.

[1912] Emotion Recognition: Use the EmotionRecognizer library to recognize the user's emotional state by analyzing their facial expressions and voice.

[1913] API communication: Use APIConnector to register information on the designated listing platform.

[1914] Specific examples

[1915] The following processing procedure will explain a specific example of when a user registers a part.

[1916] Program processing

[1917] 1. Obtaining product images:

[1918] Users take a photo of the part using a smartphone or dedicated camera and upload the image to the system.

[1919] 2. Image analysis and product information identification:

[1920] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information, which is then used to identify the part.

[1921] 3. Get part information:

[1922] Based on the identified part information, the server uses the DatabaseConnector to retrieve detailed part information from an existing product database, including part number, name, characteristics, etc.

[1923] 4. View automatically generated registration information:

[1924] Based on the acquired part information, the server automatically generates registration information such as registration title, description, and initial inventory, and displays it to the user.

[1925] 5. Emotion recognition:

[1926] It uses a face detection camera to capture the user's facial expressions and analyzes their emotional state using the EmotionRecognizer library, and the analysis results are sent to the server.

[1927] 6. Dynamic Adjustment:

[1928] The server dynamically adjusts information input and the interface depending on the user's emotional state, displaying additional assistance messages and tooltips if the user feels anxious or uncertain.

[1929] 7. Completion and final confirmation:

[1930] The user enters any missing information (e.g., arrival date and part condition) and performs a final check.

[1931] 8. Listing Information:

[1932] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[1933] Prompt Sentence Examples

[1934] Below is an example of a prompt that can be used to register a part:

[1935] Prompt 1:

[1936] To register a part, take a photo of the part and upload it. Once the photo is uploaded, the system will recognize the part and automatically generate the registration information. If you have any problems or questions during the process, an assistance message will be displayed, so please follow the instructions.

[1937] response:

[1938] The part was successfully recognized. The following information was automatically generated:

[1939] Title: Part Name

[1940] Description: Part description

[1941] Initial stock: 1

[1942] Please enter any missing information (such as arrival date) and make a final confirmation.

[1943] This allows the interface to adapt to the user's emotional state, making the registration process more efficient and stress-free.

[1944] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1945] Step 1:

[1946] Users take photos of products or parts using a smartphone or dedicated camera and upload the images to the system.

[1947] Input: Product image taken by the user

[1948] Output: Product image data uploaded to the system

[1949] Specific operation: The user launches the camera app, takes a picture of the product, and then sends the image file to the server using the specified upload button.

[1950] Step 2:

[1951] The server receives the uploaded image, analyzes it using OpenCV and Tesseract, and extracts text information from the product image.

[1952] Input: Uploaded product image data

[1953] Output: Text information extracted using OCR technology (e.g., part number, product name)

[1954] Specific operation: The server reads the image file, preprocesses it with OpenCV, and extracts text using Tesseract.

[1955] Step 3:

[1956] The server uses the DatabaseConnector to search an existing product database based on the extracted character information and obtains detailed part information.

[1957] Input: Character information extracted using OCR technology

[1958] Output: Detailed part information (e.g. part name, features, specs) retrieved from product database

[1959] Specific operation: The server uses the text information as a search query and DatabaseConnector to retrieve data on related parts from the database.

[1960] Step 4:

[1961] Based on the part information acquired by the server, registration information such as registration title, description, and initial inventory is automatically generated and sent to the terminal.

[1962] Input: Detailed part information retrieved from the database

[1963] Output: Auto-generated registration information

[1964] Specific operation: The server analyzes the part information, automatically formats the necessary registration information (title, description, inventory, etc.) using a template, and sends it to the user's device.

[1965] Step 5:

[1966] The terminal displays the automatically generated registration information to the user and prompts the user to enter any missing information as necessary.

[1967] Input: Auto-generated registration information sent from the server

[1968] Output: Final registration information including any supplementary information entered by the user

[1969] Specific operation: The device displays the registration information on the screen and provides a format (e.g., text entry fields) in which the user can enter any missing information.

[1970] Step 6:

[1971] It uses a face detection camera to capture the user's facial expressions, analyzes their emotional state using the EmotionRecognizer library, and sends the analysis results to the server.

[1972] Input: User facial expression capture image

[1973] Output: User's emotional state data

[1974] Specific operation: The device's camera periodically captures the user's facial expressions, inputs the data into the EmotionRecognizer library, analyzes the emotional state, and sends it to the server.

[1975] Step 7:

[1976] The server dynamically adjusts information input and interface according to the user's emotional state.

[1977] Input: User's emotional state data

[1978] Output: Dynamically adjusted interface or assist message

[1979] Specific operation: The server analyzes the received emotional state data, and if the user feels anxious or uncertain, it generates additional assistance messages or tooltips and displays them on the device.

[1980] Step 8:

[1981] The user enters any missing information and performs a final check.

[1982] Input: Missing information (e.g., arrival date, part condition)

[1983] Output: Final verified complete registration information

[1984] Specific operation: The user enters the required information and presses the final confirmation button, which sends the information to the server.

[1985] Step 9:

[1986] Based on all the information entered, the server uses APIConnector to register the part information on the designated listing platform, completing the registration procedure.

[1987] Input: Complete and final registration information

[1988] Output: Notification of completion of registration to the listing platform

[1989] What happens: The server consolidates all the information, uses the APIConnector to send the necessary data to the listing platform's API, and generates a notification to complete the registration process.

[1990] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1991] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1992] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1993] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1994] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1995] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1996] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1997] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1998] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1999] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2000] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2001] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2002] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2003] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2004] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2005] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2006] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2007] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2008] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2009] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2010] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2011] The following is further disclosed regarding the above embodiment.

[2012] (Claim 1)

[2013] A means for acquiring product images;

[2014] A means for analyzing the acquired image and identifying product information;

[2015] means for automatically generating listing information based on the identified product information;

[2016] a means for prompting a user to input missing listing information;

[2017] A means for listing the listing information on a predetermined platform;

[2018] A system including:

[2019] (Claim 2)

[2020] The system of claim 1, wherein product information is identified using image recognition technology.

[2021] (Claim 3)

[2022] 10. The system of claim 1, wherein an existing product database is utilized when generating listing information.

[2023] (Claim 4)

[2024] 10. The system of claim 1, further comprising means for retrieving product information based on product features extracted from the image.

[2025] (Claim 5)

[2026] 2. The system according to claim 1, further comprising means for prompting confirmation and completion of missing information via a user interface.

[2027] (Claim 6)

[2028] 10. The system of claim 1, further comprising means for automatically performing the listing process using an API of the listing platform.

[2029] "Example 1"

[2030] (Claim 1)

[2031] A means for acquiring product images;

[2032] A means for uploading the acquired images to a server;

[2033] A means for analyzing uploaded images and identifying product information using image recognition technology and optical character recognition technology (OCR);

[2034] A means for searching an existing product database based on the identified product information to acquire product information;

[2035] A means for automatically generating listing information based on the acquired product information;

[2036] a means for prompting a user to input missing listing information;

[2037] A means for listing the listing information on a predetermined platform;

[2038] A system including:

[2039] (Claim 2)

[2040] The system of claim 1, wherein product information is identified using image recognition technology and character recognition technology.

[2041] (Claim 3)

[2042] 10. The system of claim 1, wherein an existing product database is utilized when generating listing information.

[2043] "Application Example 1"

[2044] New Claims: Version with Application Examples

[2045] (Claim 1)

[2046] A means for acquiring product images;

[2047] A means for analyzing the acquired image and identifying product information;

[2048] means for automatically generating listing information based on the identified product information;

[2049] a means for prompting a user to input missing listing information;

[2050] A means for listing the listing information on a predetermined platform;

[2051] a means for suggesting prices based on user-entered criteria;

[2052] A means for automatically linking the generated listing information to a predetermined e-commerce platform;

[2053] A system including:

[2054] (Claim 2)

[2055] The system of claim 1, wherein the product information is identified using image recognition technology and the product information is obtained from an external database.

[2056] (Claim 3)

[2057] The system according to claim 1, wherein when generating the listing information, an appropriate description and title are generated based on the product condition and accessory information input by the user.

[2058] "Example 2: Combining Emotion Engines"

[2059] (Claim 1)

[2060] A means for acquiring product images using a photographing device;

[2061] A means for analyzing the acquired image and identifying product information;

[2062] means for automatically generating listing information based on the identified product information;

[2063] means for recognizing a user's emotional state using an emotion engine and dynamically adjusting the interface;

[2064] a means for prompting a user to input missing listing information;

[2065] A means for listing the generated listing information on a predetermined network platform;

[2066] A system including:

[2067] (Claim 2)

[2068] 10. The system of claim 1, wherein the system identifies product information using image recognition technology and optical character recognition.

[2069] (Claim 3)

[2070] 2. The system of claim 1, wherein an existing product database is utilized when automatically generating listing information.

[2071] "Application example 2 when combining emotion engines"

[2072] (Claim 1)

[2073] A means for acquiring product images;

[2074] A means for analyzing the acquired image and identifying product information;

[2075] means for automatically generating listing information based on the identified product information;

[2076] a means for prompting a user to input missing listing information;

[2077] A means for listing the listing information on a predetermined platform;

[2078] A means for recognizing a user's emotions and dynamically adjusting information input and an interface according to the emotions;

[2079] A system including:

[2080] (Claim 2)

[2081] The system of claim 1, wherein product information is identified using image recognition technology.

[2082] (Claim 3)

[2083] 10. The system of claim 1, wherein an existing product database is utilized when generating listing information. [Explanation of symbols]

[2084] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring product images; A means for analyzing the acquired image and identifying product information; means for automatically generating listing information based on the identified product information; a means for prompting a user to input missing listing information; A means for listing the listing information on a predetermined platform; A system including:

2. The system of claim 1 , wherein the product information is identified using image recognition technology.

3. 10. The system of claim 1, wherein an existing product database is utilized when generating the listing information.

4. 2. The system of claim 1, further comprising means for retrieving product information based on product features extracted from the image.

5. 2. The system according to claim 1, further comprising means for prompting confirmation and completion of missing information via a user interface.

6. The system of claim 1, further comprising means for automatically carrying out the listing procedure using an API of the listing platform.

Images