System

Abstract

To provide a system.SOLUTION: A system comprises: means for acquiring image data including an image of an article transmitted from a terminal device; means for analyzing the image data to identify the type, brand, model, and condition of the article; means for calculating a transaction price in the secondary distribution market of the article on the basis of the identified type, brand, model, and condition of the article; and means for transmitting the calculated transaction price to the terminal device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] It is said that currently, the average person in Japan has hidden assets of 281,277 yen. However, in order to effectively utilize these assets, there are problems such as how much each item can be sold for and how troublesome it is to list them. [Means for solving the problem]

[0005] This invention uses image analysis to calculate the secondary market price of an item and displays it in AR (Augmented Reality), allowing users to grasp their assets in real time. It also provides a means to list items based on the displayed market price, reducing the burden of listing work. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16]FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, a user takes a picture of an object they own using a camera such as a smartphone. The captured image is analyzed by an image analysis means to determine the type and condition of the object. Based on the results of this determination, the market price of the object in secondary distribution is calculated. The calculated market price is displayed to the user using AR (augmented reality) technology. Specifically, when the user looks at an object through the smartphone camera, the market price is displayed above the object.

[0029] "Example 2"

[0030] Furthermore, users can list items based on the displayed market price. Specifically, when a user taps on an item with a market price displayed on the smartphone screen, the listing screen for that item is displayed. On this screen, users can enter the listing price and listing comment, and then press the listing button to list the item.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The user takes a photo of an object they own using a smartphone or other camera.

[0034] Step 2: The captured image is analyzed by an image analysis means to determine the type and condition of the object.

[0035] Step 3: Based on the results of the assessment, the market price for the item in secondary distribution is calculated.

[0036] Step 4: The calculated price is displayed to the user using augmented reality (AR) technology. Specifically, when the user looks at an object through the smartphone camera, the price is displayed on top of the object.

[0037] "Example 2"

[0038] Step 1: The user taps on an object on their smartphone screen that displays a market price.

[0039] Step 2: When you tap, the listing screen for that item will appear.

[0040] Step 3: On this screen, the user enters the listing price and listing comments.

[0041] Step 4: You can list your item by pressing the list button.

[0042] Example 1

[0043] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0044] With conventional systems, it was difficult for users to quickly and accurately grasp the market price of the items they owned in the secondary distribution market. In addition, there was a lack of a way to display market price information in an easy-to-understand visual format, so users could not fully utilize it as a basis for making decisions when listing items. This made it difficult for users to buy and sell items at the appropriate price, resulting in a problem of reduced efficiency in transactions.

[0045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0046] In this invention, the server includes a means for a user to photograph an object using a photographing device, a means for transmitting the photographed image to an analysis device, a means for the analysis device to determine the type and condition of the object using image analysis technology, a means for calculating the market price of the object in the secondary distribution market based on the determination result, a means for transmitting the calculated market price to a display device, and a means for the display device to display the market price using augmented reality technology. This allows the user to quickly and accurately grasp the market price of the object they own and to confirm it in a visually easy-to-understand manner. In addition, the user can list the object at an appropriate price, improving the efficiency of transactions.

[0047] A "user" is an individual or corporation that uses the system to photograph items, check market prices, and list them for sale.

[0048] An "imaging device" is a device such as a camera or smartphone that a user uses to photograph an object.

[0049] An "analysis device" is a computer system that receives captured images and uses image analysis technology to determine the type and condition of an object.

[0050] "Image analysis technology" refers to algorithms and software for analyzing image data to determine the type and condition of an object.

[0051] A "secondary market" is a market in which goods are bought and sold again after they have been in the hands of consumers.

[0052] "Market price" refers to the general trading price of a product in the secondary distribution market.

[0053] A "display device" is a device for visually displaying the calculated market price to a user, and includes a smartphone, tablet, etc.

[0054] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[0055] "Listing" refers to the act of a user offering something they own on the market for sale.

[0056] This invention is a system that quickly and accurately grasps the market price in the secondary distribution market of an item owned by a user and displays it in a visually easy-to-understand manner. A specific embodiment of this system is described below.

[0057] A user takes a picture of an object using a camera such as a smartphone. The captured image is sent to an analysis device through an application on the smartphone. The analysis device uses image analysis technology such as Google® Cloud Vision API to determine the type and condition of the object in the image. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch may be determined.

[0058] The analysis device calculates the market price of the item in the secondary distribution market based on the judgment result. Specifically, it refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis result determines that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[0059] The calculated market price is sent from the analysis device to a smartphone. The smartphone displays the received market price using AR (Augmented Reality) technology. Specifically, when a user looks at an object through the smartphone's camera, the market price is displayed above the object. For example, when a user looks at a watch through the camera, "Market price: \10,000" is displayed above the watch. Libraries such as ARKit (iOS) and ARCore (ANDROID (registered trademark)) are used for AR display.

[0060] This system allows users to quickly and accurately grasp the market price of their items and check it in a visually easy-to-understand format. It also allows users to list their items at an appropriate price, improving the efficiency of transactions.

[0061] As a concrete example, consider the case where a user has an old Seiko watch. The user activates the camera on their smartphone and takes a picture of the watch. The captured image is sent from the smartphone to a server, where the brand, model, and condition of the watch are analyzed using the Google Cloud Vision API. Based on the analysis results, a market price is calculated using the eBay API and past transaction prices of similar watches. The calculated market price is sent to the smartphone, and when the user looks at the watch through the camera, "Market Price: \10,000" is displayed above the watch.

[0062] An example of a prompt sentence might be:

[0063] "Write a program that takes a picture of an old Seiko watch, analyzes the brand, model and condition of the watch using the Google Cloud Vision API, calculates a market price by referencing the eBay API, and uses AR technology to display the market price on top of the watch through the smartphone camera."

[0064] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0065] Step 1:

[0066] The user takes a picture of an object using the smartphone camera.

[0067] Input: An image of an object taken by the user.

[0068] Specific operation: The user launches the smartphone camera app and takes a picture of an object they own. For example, if the user has an old watch, they can take a picture of the watch with the camera. When they press the capture button, the image is saved on the smartphone.

[0069] Output: The captured image data.

[0070] Step 2:

[0071] The terminal transmits the captured image to the server.

[0072] Input: The captured image data.

[0073] Specific operation: A smartphone application uploads image data to a server using an HTTP request. For example, the image data is encoded in Base64 format and sent in the body of a POST request.

[0074] Output: Image data sent to the server.

[0075] Step 3:

[0076] The server performs image analysis.

[0077] Input: Image data sent to the server.

[0078] How it works: The server uses image analysis technology such as the Google Cloud Vision API to determine the type and condition of objects in the image. The server sends image data to the API and receives the analysis results in return. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch are determined.

[0079] Output: Analysis results regarding the type and condition of the item.

[0080] Step 4:

[0081] The server calculates the market price of the item in the secondary distribution market.

[0082] Input: Analysis results regarding the type and condition of the item.

[0083] Specific operation: Based on the analysis results, the server refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis results determine that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[0084] Output: The calculated market price.

[0085] Step 5:

[0086] The server transmits the market price to the terminal.

[0087] Input: The calculated market price.

[0088] Specific operation: The server generates JSON data including the market price and returns it to the terminal as an HTTP response. For example, it returns data in the format {"item": "Seiko Watch", "price": "\10,000"}.

[0089] Output: Quote price data sent to the terminal.

[0090] Step 6:

[0091] The device displays market prices using AR technology.

[0092] Input: Quote price data sent to the terminal.

[0093] Specific operation: The smartphone uses AR (Augmented Reality) technology to display the received market price. Specifically, the market price is overlaid on the user's smartphone camera view. For example, when the user looks at their watch through the camera, "Market price: ¥10,000" is displayed on top of the watch. Libraries such as ARKit (iOS) and ARCore (Android) are used for the AR display.

[0094] Output: Quoted prices displayed using AR technology.

[0095] (Application example 1)

[0096] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] With conventional systems, it was difficult for users to easily grasp the market price of their items in the secondary market. In addition, in order to check the market price, users had to use multiple websites and applications, which was time-consuming. Furthermore, there was a lack of a way to check the market price in real time when appraising or purchasing items in a physical store.

[0098] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating the market price of the item in the secondary distribution based on the image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing the item based on the displayed market price, a means for photographing the item using the camera of the smart device and determining the type and condition of the item using image analysis technology, a means for acquiring market price data from a database based on the determination result and calculating the market price, a means for displaying the calculated market price on the screen of the smart device using AR technology, and a means for generating a prompt sentence for calculating the market price using a generative AI model. This allows the user to easily check the market price of the item using the smart device and grasp the market price in real time when the item is appraised or purchased in a physical store.

[0099] "Image analysis means" is a technology for analyzing image data acquired through a camera and determining the type and condition of an object.

[0100] The "market price calculation means" is a technology for calculating the market price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[0101] "Means of displaying using AR (Augmented Reality)" refers to technology for overlaying the calculated market price on the screen of a smart device.

[0102] A "means of listing" refers to technology that allows an item to be listed on an online marketplace, etc., based on user operations.

[0103] "Smart device" is a general term for portable electronic devices such as smartphones and tablets that can connect to the Internet and have a camera function.

[0104] "Image analysis technology" is a technology for analyzing image data captured by a camera and extracting specific information.

[0105] "Database" means an information system that stores market data and other related information in an organized manner and enables it to be searched and retrieved as needed.

[0106] A “generative AI model” is a model that is trained to perform a specific task using artificial intelligence, such as in this case to generate prompts for calculating quotes.

[0107] A "prompt sentence" is an input sentence that causes a generative AI model to perform a specific task.

[0108] A system for implementing the present invention uses the following hardware and software.

[0109] Hardware used

[0110] Smart devices (smartphones, tablets, etc.)

[0111] Server (image analysis, database)

[0112] Software used

[0113] Image analysis library (e.g. OpenCV)

[0114] AR library (e.g. ARCore)

[0115] Database (e.g. MySQL (registered trademark))

[0116] Generative AI models (e.g. GPT-4 (registered trademark))

[0117] System configuration and operation

[0118] 1. Image capture

[0119] The user uses the camera on their smart device to take a picture of the item they want to appraise, and the image data is temporarily stored on the smart device.

[0120] 2. Image Analysis

[0121] The smart device sends the captured image data to the server, which uses an image analysis library (e.g., OpenCV) to analyze the image data and determine the type and condition of the product.

[0122] 3. Market Calculation

[0123] Based on the results of image analysis, the server retrieves market price data for the relevant item in the secondary distribution market from a database (e.g. MySQL), and calculates the market price based on the retrieved data.

[0124] 4. AR display

[0125] The calculated market price is sent from the server to the smart device, which uses an AR library (e.g., ARCore) to display the market price overlaid on the product seen through the camera.

[0126] 5. Use of generative AI models

[0127] The server uses a generative AI model (e.g. GPT-4) to generate a prompt to calculate the market price. The generated prompt has the following format:

[0128] "What type of item is this, and what is the current going price on the secondary market?"

[0129] Examples

[0130] For example, consider the case where a user wants to purchase a used smartphone at a physical store. The user takes a picture of the product using the smartphone camera. The captured image is sent to the server and analyzed using OpenCV. Based on the analysis results, the market price of the same model in the secondary market is obtained from the database. The calculated market price is displayed on the smartphone screen using ARCore. In this way, the user can easily check the market price when purchasing the product at a physical store.

[0131] This system allows users to easily check the market price of items using their smart devices, and understand the real-time market price when having items appraised or purchased in a physical store.

[0132] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0133] Step 1:

[0134] The user uses the camera of the smart device to take a picture of the item to be appraised. The input is the product image, and the output is the image data temporarily stored in the smart device. In concrete terms, the user starts the camera app, frames the product, and presses the shutter button.

[0135] Step 2:

[0136] The terminal transmits the captured image data to the server. The input is the temporarily stored image data, and the output is the image data transmitted to the server. In concrete terms, the terminal uploads the image data to the server via an Internet connection.

[0137] Step 3:

[0138] The server uses an image analysis library (e.g. OpenCV) to analyze the image data and determine the type and condition of the product. The input is the image data sent to the server, and the output is data about the type and condition of the product. Specifically, the server runs an image analysis algorithm to extract features in the image and identify the product.

[0139] Step 4:

[0140] Based on the results of image analysis, the server retrieves secondary market price data for the relevant product from a database (e.g. MySQL). The input is data about the type and condition of the product, and the output is price data. Specifically, the server executes a database query to search for price information for the relevant product.

[0141] Step 5:

[0142] The server calculates the market price based on the acquired market price data. The input is the market price data, and the output is the calculated market price. Specifically, the server uses statistical methods and algorithms to analyze the market price data and calculate the average price and median price.

[0143] Step 6:

[0144] The server transmits the calculated market price to the smart device. The input is the calculated market price, and the output is the market price transmitted to the smart device. In particular, the server transmits market price data to the smart device via an internet connection.

[0145] Step 7:

[0146] The device uses an AR library (e.g. ARCore) to overlay the market price on the product seen through the camera. The input is the market price sent from the server, and the output is the market price displayed on the smart device screen. In concrete terms, the device overlays the market price on the camera image.

[0147] Step 8:

[0148] The server uses a generative AI model (e.g. GPT-4) to generate a prompt for calculating the market price. The input is data on the type and condition of the product, and the output is the generated prompt. In concrete terms, the server requests the generative AI model to generate a prompt, and obtains the following prompt:

[0149] "What type of item is this, and what is the current going price on the secondary market?"

[0150] In this way, users can easily check the market price of items using their smart devices and understand the market price in real time when having an item appraised or purchased in a physical store.

[0151] Example 2

[0152] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0153] With conventional systems, it was difficult to accurately calculate the market price of goods in the secondary distribution market and provide an environment where users could easily list items for sale. In addition, there was a lack of a way to effectively utilize past transaction data when calculating market prices, so it was not possible to provide users with reliable market price information. Furthermore, the listing process was complicated, placing a heavy burden on users.

[0154] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating a market price in the secondary distribution of an item based on image data obtained by the image analysis means, a means for calculating a market price using a generative AI model, a means for displaying the calculated market price, a means for listing an item based on the displayed market price, a means for inputting listing information based on a user's operation, a means for transmitting the input listing information to the server, and a means for storing the listing information received by the server in a database. This enables a user to easily list an item based on highly reliable market price information.

[0155] An "image analysis means" is a device or software that analyzes image data acquired through a camera and recognizes the characteristics and condition of an object.

[0156] The "market price calculation means" refers to a device or software for calculating the price of an item in the secondary market based on the data obtained by the image analysis means and past transaction data.

[0157] A "generative AI model" is a model that uses artificial intelligence techniques to analyze data and perform specific tasks, in this case to calculate quotes.

[0158] A "means for displaying quotes" is a device or software for visually presenting the calculated quotes to a user.

[0159] A "means for listing" is a device or software that supports a series of operations for a user to list an item on the secondary market.

[0160] "Means for inputting listing information" refers to a device or software that allows a user to input information such as the price and comments of the item being listed.

[0161] The "means for transmitting the selling information to the server" refers to a device or software for transmitting the selling information input by the user to the server.

[0162] The "means for storing in a database" refers to a device or software that stores the listing information received by the server in a database and makes it available for other users to view.

[0163] This invention is a system that allows a user to list items for sale using a smartphone, and includes an image analysis means, a means for calculating a market price, a generative AI model, a means for displaying a market price, a means for listing an item, a means for inputting listing information, a means for transmitting the listing information to a server, and a means for saving the listing information in a database.

[0164] The user starts the app on their smartphone and uses the camera to capture an image of the item they wish to sell. The image analysis means analyzes the captured image data and recognizes the characteristics and condition of the item. The analyzed data is then sent to the server.

[0165] The server retrieves historical trading data from a database (e.g., MySQL) and calculates quotes using a generative AI model (e.g., GPT-4 by OpenAI). For example, it sends a prompt like the following to the generative AI model:

[0166] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[0167] Transaction 1: 400 yen

[0168] Transaction 2: 500 yen

[0169] Transaction 3: 600 yen

[0170] The generative AI model calculates the market price based on this data, and the server sends the calculated market price to the user's device.

[0171] The terminal displays the received market price on the screen. For example, it may say, "The market price for this book is 500 yen." The user selects the item to sell based on the displayed market price, and inputs the selling price and listing comment on the listing screen. For example, the user sets the selling price to 600 yen and inputs the comment, "The item is in good condition."

[0172] When the user presses the sell button, the terminal sends the entered selling information to the server, which then stores the received selling information in a database and makes it available for other users to view.

[0173] The system allows users to easily list items based on reliable market information, and by using generative AI models, it effectively leverages past transaction data to provide accurate market prices.

[0174] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0175] Step 1:

[0176] The user launches the app.

[0177] A user launches an app by tapping the app icon on the home screen of their smartphone. The input is the user's operation, and the output is the launch of the app.

[0178] Step 2:

[0179] A user uses a camera to capture an image of an object.

[0180] A user uses the camera function in the app to take a picture of the item they want to put up for sale. The input is the image data acquired through the camera, and the output is the image data sent to the image analysis means.

[0181] Step 3:

[0182] The server analyzes the image data.

[0183] The server uses image analysis means to analyze the acquired image data and recognize the features and state of the object. The input is the image data sent by the user, and the output is the analyzed feature data of the object.

[0184] Step 4:

[0185] The server calculates the market price.

[0186] The server retrieves historical trading data from the database and uses the generative AI model to calculate quotes. For example, it sends the following prompt to the generative AI model:

[0187] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[0188] Transaction 1: 400 yen

[0189] Transaction 2: 500 yen

[0190] Transaction 3: 600 yen

[0191] The input is historical trading data and a prompt, and the output is the calculated quote price.

[0192] Step 5:

[0193] The server transmits the market price to the terminal.

[0194] The server sends the calculated quotes to the user's terminal. The input is the calculated quote and the output is the quote sent to the terminal.

[0195] Step 6:

[0196] The terminal will display the market price.

[0197] The terminal displays the received market price on the screen. For example, it displays "The market price of this book is 500 yen." The input is the market price sent from the server, and the output is the market price displayed on the screen.

[0198] Step 7:

[0199] The user selects the item to sell.

[0200] The user taps the item they want to sell from the list of market prices displayed on the screen. The input is the user's operation, and the output is a detailed screen of the selected item.

[0201] Step 8:

[0202] The device displays the listing screen.

[0203] The terminal opens a listing screen that displays detailed information about the selected item. The input is the information about the item selected by the user, and the output is the listing screen.

[0204] Step 9:

[0205] The user enters listing information.

[0206] The user inputs the listing price and listing comment on the listing screen. For example, the listing price is set to 600 yen and the comment "Condition is good." The input is the listing information entered by the user, and the output is the entered listing information.

[0207] Step 10:

[0208] The user presses the sell button.

[0209] The user presses the "list" button to confirm the information they have entered. The input is the user's operation, and the output is the confirmed listing information.

[0210] Step 11:

[0211] The terminal transmits the listing information to the server.

[0212] The terminal transmits the inputted selling information to the server. The input is the confirmed selling information, and the output is the selling information transmitted to the server.

[0213] Step 12:

[0214] The server stores the listing information in a database.

[0215] The server stores the received listings in a database and makes them available for other users to view. The input is the listing sent to the server and the output is the listing stored in the database.

[0216] (Application example 2)

[0217] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] In the conventional secondary distribution of goods, it was difficult for users to grasp the appropriate market price, and the listing process was cumbersome. In addition, it was time-consuming for users to input the listing price and comments, and the listing information was not saved efficiently. This reduced user convenience and hindered the revitalization of the secondary distribution market.

[0219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0220] In this invention, the server includes an image analysis means, a means for calculating a market price for an item in secondary distribution based on image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing an item based on the displayed market price, and a means for a user to input a listing price and a comment and store listing information. This allows a user to easily grasp an appropriate market price and efficiently list items. Furthermore, since the listing information is efficiently stored, user convenience is improved and the secondary distribution market is expected to be revitalized.

[0221] An "image analysis means" is a device or software that analyzes image data acquired through a camera and extracts features and attributes of objects.

[0222] The "market price calculation means" refers to a device or software that calculates the fair price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[0223] "Means of displaying using AR (Augmented Reality)" refers to a technology that displays the calculated market price by overlaying it on a real-world image, and is a device or software that allows users to visually confirm the market price.

[0224] "Means for listing" refers to a device or software that provides an interface for users to list items based on market prices and allows for the input and storage of listing information.

[0225] The "means for inputting the listing price and comments and storing the listing information" refers to a device or software that allows the user to input the listing price and comments and stores the information in a database or the like.

[0226] A system for carrying out this invention has the following configuration. First, a user takes an image of an object using a smartphone. The smartphone is equipped with a camera, and image data acquired through this camera is sent to image analysis means. The image analysis means analyzes the image data and extracts features and attributes of the object.

[0227] Next, based on the data obtained by the image analysis means, the server calculates the fair price of the item in the secondary distribution market. This is the means of calculating the market price. The calculated market price is displayed on the smartphone display using AR (Augmented Reality) technology. This allows the user to visually check the market price by overlaying it on an image of the real world.

[0228] After the user checks the market price, an interface for entering the listing price and comments will be displayed on the smartphone screen. The user enters the listing price and comments through this interface and saves the listing information by pressing the listing button. This is the means for listing and saving the listing information.

[0229] The hardware used is a smartphone, and the software uses Flask (a Python web framework). Data processing and calculation involve obtaining market prices and storing listing information. Specifically, computer vision technology is used for image analysis, and machine learning models are used to calculate market prices.

[0230] As a concrete example, the prompt text when a user lists "item1" is as follows:

[0231] Example prompt:

[0232] A user is trying to list "item1". Get the market price, set the listing price to 1500 yen, and set the comment to "Brand new, unused."

[0233] In this way, users can easily check market prices and efficiently list items for sale. Furthermore, because listing information is efficiently stored, user convenience is improved and the secondary distribution market is expected to become more active.

[0234] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0235] Step 1:

[0236] A user takes an image of an object using the smartphone camera.

[0237] Input: Image of thing

[0238] Output: Image data

[0239] Specific operation: A user starts the camera app on their smartphone and takes a picture of an object. The captured image data is saved in the smartphone.

[0240] Step 2:

[0241] The terminal uses image analysis means to analyze the captured image data.

[0242] Input: Image data

[0243] Output: Feature and attribute data of objects

[0244] How it works: Image analysis software in the smartphone processes image data and extracts the object's features (e.g., shape, color, size) and attributes (e.g., brand, condition).

[0245] Step 3:

[0246] The server calculates the market price of the item in the secondary distribution market based on the data obtained by the image analysis means.

[0247] Input: Features and attribute data of objects

[0248] Output: Market price data

[0249] Specific operation: The server uses a machine learning model to input the item's characteristics and attributes, and calculates the market price based on past transaction data and market data.

[0250] Step 4:

[0251] The device displays the calculated market price using AR (augmented reality) technology.

[0252] Input: Market price data

[0253] Output: AR displayed market price

[0254] How it works: The smartphone's AR app receives market price data and displays the market price by overlaying it on real-world images via the camera.

[0255] Step 5:

[0256] The user enters the listing price and comments on the smartphone screen.

[0257] Input: Market price data, user-entered listing prices and comments

[0258] Output: Listing information

[0259] Specific operation: The user enters the listing price and comments into the listing interface on the smartphone and presses the listing button. The entered information is compiled into the listing information.

[0260] Step 6:

[0261] The server stores the listing information.

[0262] Input: Listing information

[0263] Output: Saved listings

[0264] Specific operation: The server saves the listing information to the database and notifies the user that the listing is complete.

[0265] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0266] "Example 1"

[0267] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine recognizes emotions from the user's facial expressions and tone of voice. Specifically, the user takes a picture of his or her face through the smartphone camera, and recognizes emotions such as joy, sadness, surprise, and anger from the facial expressions. In addition, the user records his or her voice through the smartphone microphone, and recognizes emotions from the tone of the voice.

[0268] "Example 2"

[0269] Furthermore, the emotion engine recommends listing items based on the user's emotions. Specifically, if a user expresses an emotion of happiness, it recommends listing items that make the user happy. Conversely, if a user expresses an emotion of sadness, it recommends listing items that make the user sad. In this way, the emotion engine understands the user's emotions and helps the user list items based on them.

[0270] The process flow of each embodiment will be described below.

[0271] "Example 1"

[0272] Step 1: The user takes a picture of their face using their smartphone camera.

[0273] Step 2: The emotion engine recognizes emotions such as happiness, sadness, surprise, and anger from the captured facial expressions.

[0274] Step 3: The user records their voice through the smartphone's microphone.

[0275] Step 4: The emotion engine identifies emotions from the tone of the recorded voice.

[0276] "Example 2"

[0277] Step 1: The emotion engine recognizes the user's emotion.

[0278] Step 2: If the user expresses the emotion of joy, the emotion engine will recommend listing something that brings the user joy.

[0279] Step 3: If the user indicates the emotion of sadness, the emotion engine will recommend that the user list something that makes them sad.

[0280] Example 1

[0281] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0282] In the conventional system, it was difficult for users to easily grasp the secondary market price of the items they owned, and the information provided did not take into account the users' feelings. As a result, it was difficult for users to make decisions when listing items, and there was an issue that the user experience did not improve.

[0283] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0284] In this invention, the server includes an image analysis means, a means for calculating the secondary market price of the item, a means for displaying the market price in augmented reality, a means for recognizing the user's emotions from their facial expressions and tone of voice, and a means for providing information to the user based on the recognized emotions. This allows the user to easily grasp the secondary market price of the item they own, and to receive appropriate information according to their emotions.

[0285] The "image analysis means" is a means for analyzing image data acquired through a camera and determining the type and condition of an object.

[0286] The "means for calculating the market price in the secondary distribution of goods" refers to a means for calculating the trading price in the secondary distribution market of goods based on the data obtained by the image analysis means.

[0287] "Means for displaying using augmented reality" refers to means for visually displaying the calculated market price to the user using augmented reality technology.

[0288] "Means for recognizing emotions from a user's facial expressions and tone of voice" refers to means for analyzing a user's facial expressions and tone of voice to identify emotions.

[0289] The "means for providing information to a user based on a recognized emotion" refers to a means for providing appropriate information to a user in accordance with the emotion of the user identified by the emotion recognition means.

[0290] A "means for listing items" is a means for a user to list items they own on the secondary market based on the displayed market price.

[0291] The present invention is a system that allows a user to easily grasp the secondary market price of an item owned by the user and provides information according to the user's emotions. A specific embodiment of this system will be described below.

[0292] System Configuration

[0293] This system consists of a user's terminal (smartphone), a server, an image analysis means, a market price calculation means, an augmented reality display means, an emotion recognition means, an information provision means, and an listing means.

[0294] Hardware and Software

[0295] Device (smartphone): Camera, microphone, AR app (e.g. ARKit, ARCore)

[0296] Server: Image analysis software (e.g. Google Cloud Vision API), market price calculation software (e.g. eBay API, Amazon API), emotion recognition software (e.g. Microsoft® Azure® Emotion API)

[0297] Processing flow

[0298] 1. The user takes a photo of an object

[0299] Users use the smartphone camera to take a picture of an object they own. For example, if a user wants to take a picture of an old book, they start the smartphone camera app and take a picture of the book cover.

[0300] 2. The device sends the image to the server

[0301] The device sends the captured image to the server. At this time, the device uses an Internet connection to upload the image data to the server.

[0302] 3. The server analyzes the image

[0303] The server analyzes the received images. Specifically, the server uses the Google Cloud Vision API to determine the type and condition of the objects in the image. For example, the server can identify the title, author, and condition of a book (new, used, damaged, etc.).

[0304] 4. The server calculates the secondary market price of the item

[0305] Based on the results of the search, the server calculates the market price of the item in the secondary market. To do this, the server refers to the eBay API or Amazon API to check the past transaction prices of similar items. For example, the server calculates the market price by averaging the transaction prices of the same book over the past year.

[0306] 5. The server sends the quotes to the terminal

[0307] The server then sends the calculated market price to the device, which receives the data and displays the market price using an AR app.

[0308] 6. The device displays market prices in AR

[0309] The device uses ARKit and ARCore to display market prices on objects when a user looks at them through the smartphone camera. For example, when a user looks at a book through the camera, "Market price: $1,500" is displayed on the book.

[0310] 7. Recognize user emotions

[0311] The user uses the smartphone camera to take a picture of their face and the smartphone microphone to record their voice, and the device transmits this data to the server.

[0312] 8. The server analyzes emotions

[0313] The server uses the Microsoft Azure Emotion API to identify emotions from the user's facial expressions. The server also recognizes emotions from the tone of the recorded voice. For example, the server recognizes a user's smile as "happiness" and a high-pitched voice as "excitement."

[0314] 9. The server sends the emotion recognition results to the device.

[0315] The server sends the emotion recognition results to the device, which receives the data and displays it to the user.

[0316] 10. The device displays the emotion recognition results.

[0317] The device will display the emotion recognition results to the user. For example, if the user is smiling, it will display "You are happy."

[0318] Examples

[0319] Example 1: Finding the market value of old books

[0320] 1. The user takes a photo of an old book with their smartphone camera.

[0321] 2. The device sends the image to the server.

[0322] 3. The server uses the Google Cloud Vision API to identify the book's title and status.

[0323] 4. The server calculates the market price by referencing the eBay API or Amazon API.

[0324] 5. The server sends the market price to the terminal.

[0325] 6. The device uses ARKit to display price quotes on top of the book.

[0326] Example 2: Emotion recognition

[0327] 1. The user takes a photo of their face with their smartphone camera and records their voice with the microphone.

[0328] 2. The device sends this data to the server.

[0329] 3. The server uses the Microsoft Azure Emotion API to identify emotions from facial expressions.

[0330] 4. The server recognizes emotions from the tone of the recorded voice.

[0331] 5. The server sends the emotion recognition results to the device.

[0332] 6. The device displays the emotion recognition results to the user.

[0333] Examples of prompt statements

[0334] "I want to find out the market price of an old book. I took a picture of the book with my smartphone camera. Please tell me the market price."

[0335] "Please recognize my emotions. I photographed my face with my phone camera and recorded my voice with a microphone."

[0336] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0337] Step 1:

[0338] The user takes a photo of an object

[0339] A user uses the smartphone camera to take a picture of an object they own. For example, if a user wants to take a picture of an old book, they start the smartphone camera app and take a picture of the book's cover. The input is the image of the object, and the output is the captured image data.

[0340] Step 2:

[0341] The device sends the image to the server.

[0342] The terminal sends the captured image to the server. At this time, the terminal uses an Internet connection to upload image data to the server. The input is the captured image data, and the output is the image data sent to the server.

[0343] Step 3:

[0344] The server analyzes the image

[0345] The server analyzes the images it receives. Specifically, the server uses image analysis software (e.g., Google Cloud Vision API) to determine the type and condition of the object in the image. For example, the server might identify the book's title, author, and condition (new, used, damaged, etc.). The input is the image data sent to the server, and the output is data about the type and condition of the object.

[0346] Step 4:

[0347] The server calculates the secondary market price of the item

[0348] Based on the results of the determination, the server calculates the market price of the item in the secondary market. To do this, the server refers to a database (e.g., eBay API or Amazon API) and investigates past transaction prices of similar items. For example, the server calculates the market price by averaging the transaction prices of the same book over the past year. The input is data about the type and condition of the item, and the output is the calculated market price.

[0349] Step 5:

[0350] The server sends the market price to the terminal.

[0351] The server sends the calculated market price to the terminal. The terminal receives this data and displays the market price using the AR app. The input is the calculated market price, and the output is the market price data sent to the terminal.

[0352] Step 6:

[0353] The device displays market prices in AR

[0354] The device uses ARKit and ARCore to display a current price on top of an object when the user looks at it through the smartphone camera. For example, when the user looks at a book through the camera, "Current Price: $1,500" is displayed on top of the book. The input is the current price data sent to the device, and the output is the current price displayed in AR.

[0355] Step 7:

[0356] Recognize user emotions

[0357] The user takes a picture of their face using the smartphone camera and records their voice using the smartphone microphone. The device sends these data to the server. The input is the user's face image data and voice recording data, and the output is emotion recognition data sent to the server.

[0358] Step 8:

[0359] The server analyzes emotions

[0360] The server uses emotion recognition software (e.g., Microsoft Azure Emotion API) to identify emotions from the user's facial expressions. The server also recognizes emotions from the tone of a recorded voice. For example, the server may recognize a user's smiling face as "happiness" and a high-pitched voice as "excitement." The input is the emotion recognition data sent to the server, and the output is the recognized emotion data.

[0361] Step 9:

[0362] The server sends the emotion recognition results to the device.

[0363] The server sends the emotion recognition results to the terminal. The terminal receives this data and displays it to the user. The input is the recognized emotion data, and the output is the emotion recognition result data sent to the terminal.

[0364] Step 10:

[0365] The device displays the emotion recognition results.

[0366] The terminal displays the emotion recognition result to the user. For example, if the user is smiling, "You are happy" is displayed. The input is the emotion recognition result data sent to the terminal, and the output is the emotion recognition result displayed to the user.

[0367] (Application example 1)

[0368] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0369] In the conventional secondary market, users had to spend a lot of time and effort to understand the market price of the items they own. In addition, there was also the issue that the user experience did not improve because information was provided uniformly without considering the user's feelings.

[0370] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0371] In this invention, the server includes an image analysis means, a means for calculating a market price in secondary distribution of an item based on image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing an item based on the displayed market price, an emotion recognition means for recognizing an emotion from a user's facial expression and tone of voice, and a means for providing feedback based on the recognized emotion. This allows a user to quickly and easily grasp the market price of an item that he or she owns, and to receive appropriate feedback according to the emotion.

[0372] The "image analysis means" is a means for analyzing image data acquired through a camera and determining the type and condition of an object.

[0373] The "market price calculation means" is a means for calculating the market price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[0374] "Means for displaying using AR (Augmented Reality)" refers to a means for displaying the calculated market price by overlaying it on a real image.

[0375] A "means for listing items" is a means for a user to list items on the secondary market based on the displayed market price.

[0376] An "emotion recognition means" is a means for analyzing a user's facial expressions and tone of voice to recognize the user's emotions.

[0377] The "means for providing feedback" is a means for providing appropriate feedback to a user based on the recognized user sentiment.

[0378] A system for implementing this invention has the following configuration. First, a user takes a picture of an item they own using the smartphone camera. The captured image is analyzed by image analysis means to determine the type and condition of the item. Next, the market price of the item in the secondary distribution market is calculated based on the results of the determination. The calculated market price is displayed to the user using AR (Augmented Reality) technology. Specifically, when the user looks at an item through the smartphone camera, the market price is displayed above the item.

[0379] Furthermore, an emotion recognition means is incorporated, and emotions are recognized from the user's facial expressions and tone of voice. Specifically, the user takes a picture of his / her face using the smartphone camera, and emotions such as joy, sadness, surprise, and anger are recognized from the facial expressions. In addition, the user records his / her voice using the smartphone microphone, and emotions are recognized from the tone of the voice. Based on the recognized emotions, the feedback providing means provides appropriate feedback to the user.

[0380] Hardware and software used:

[0381] Hardware: Smartphone (camera, microphone)

[0382] Software: OpenCV (image processing library), Keras (deep learning library for image analysis and emotion recognition), pyttsx3 (voice synthesis library), dlib (face detection library)

[0383] Data processing and calculation:

[0384] 1. Image analysis: Images taken with a smartphone camera are read using OpenCV, and the type and condition of the item are determined using a Keras model, and the market price in the secondary market is calculated.

[0385] 2. Emotion recognition: Face detection is performed using dlib on images of the user's face taken with a smartphone camera, and emotions are recognized using a Keras model.

[0386] 3. AR display: Using OpenCV, the calculated market price is displayed on the image.

[0387] 4. Audio feedback: Use pyttsx3 to provide audio feedback according to the recognized emotion.

[0388] Examples:

[0389] When users take a picture of their old phone with their smartphone camera, the phone's market value on the secondary market is displayed on the screen, and if the user smiles, voice feedback is provided saying, "This item has good resale value."

[0390] Example prompt for a generative AI model:

[0391] Analyze the images taken by the user using the smartphone camera and calculate the market price of the item in the secondary market. Also, recognize the emotions of the user from their facial expressions and tone of voice and provide feedback according to their emotions.

[0392] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0393] Step 1:

[0394] A user takes a picture of an object they own using the camera of their smartphone. The input is the image data acquired through the camera, and the output is that image data. In concrete terms, the user starts the camera app on their smartphone, puts the object in the frame, and presses the capture button.

[0395] Step 2:

[0396] The terminal transmits the captured image data to the image analysis means. The input is the image data acquired in step 1, and the output is the data transmitted to the image analysis means. In concrete terms, the smartphone application transmits the image data to the server.

[0397] Step 3:

[0398] The server uses image analysis means to analyze the image data and determine the type and status of the object. The input is the image data sent in step 2, and the output is data related to the type and status of the object. In concrete terms, an image analysis model (using Keras) on the server processes the image data and determines the type and status of the object.

[0399] Step 4:

[0400] The server calculates the market price of the item in the secondary market based on the judgment result. The input is the data on the type and condition of the item obtained in step 3, and the output is the market price. In concrete terms, the server refers to the database and calculates the market price based on the market prices of similar items.

[0401] Step 5:

[0402] The server sends the calculated market price to the terminal, which then displays the market price using AR technology. The input is the market price calculated in step 4, and the output is the market price displayed on the smartphone screen. In concrete terms, the smartphone application overlays the market price on the camera image.

[0403] Step 6:

[0404] A user takes a picture of his / her face through the smartphone camera, and the terminal transmits the image data to the emotion recognition means. The input is the face image data acquired through the camera, and the output is the data transmitted to the emotion recognition means. In concrete terms, the user points the smartphone camera at himself / herself and takes a picture of his / her face.

[0405] Step 7:

[0406] The server uses emotion recognition means to recognize emotions from the user's facial expressions and tone of voice. The input is the facial image data and voice data sent in step 6, and the output is the recognized emotion data. In concrete terms, the emotion recognition model (using Keras) on the server analyzes the facial image and voice data to determine the emotion.

[0407] Step 8:

[0408] The server generates feedback based on the recognized emotion and sends it to the terminal. The input is the emotion data obtained in step 7, and the output is a feedback message. In concrete terms, the server generates a feedback message according to the emotion data and sends it to the terminal.

[0409] Step 9:

[0410] The terminal provides the feedback message to the user using a speech synthesis engine. The input is the feedback message sent in step 8, and the output is the voice feedback provided to the user. In concrete terms, the smartphone application uses pyttsx3 to play back the feedback message aloud.

[0411] Example 2

[0412] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0413] In conventional secondary distribution systems, it was difficult for users to grasp the appropriate market price, and listing recommendations were not made based on the user's emotions. This made it difficult for users to list items at the optimal time, which resulted in the revitalization of the secondary distribution market being hindered. Furthermore, because listing recommendations were not made based on the user's emotions, there was a possibility that user satisfaction would decrease.

[0414] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0415] In this invention, the server includes an image analysis means, a market price calculation means, a market price display means, a listing means, an emotion analysis means, and a listing recommendation means. This allows the user to understand the appropriate market price and receive listing recommendations based on emotions. This allows the user to list items at the optimal time, promoting the revitalization of the secondary distribution market. Furthermore, listing recommendations that take the user's emotions into consideration improve user satisfaction.

[0416] The "image analysis means" refers to a means for analyzing image data acquired using a camera or other image acquisition device, and extracting necessary information.

[0417] The "market price calculation means" is a means for calculating the market price in the secondary distribution market of an item based on the data obtained by the image analysis means.

[0418] The "market price display means" is a means for visually displaying the calculated market price to the user.

[0419] A "listing means" is a means by which a user lists an item on the secondary distribution market.

[0420] The "emotion analysis means" is a means for analyzing the user's emotional data and determining the user's emotional state.

[0421] The "item recommendation means" is a means for recommending appropriate items to a user for sale based on the user's emotion data analyzed by the emotion analysis means.

[0422] This invention is a system that allows a user to grasp an appropriate market price when putting an item up for sale on a secondary market, and further allows the user to receive a recommendation for listing based on the user's emotions. This system includes an image analysis means, a market price calculation means, a market price display means, a listing means, an emotion analysis means, and a listing recommendation means.

[0423] Hardware and Software Used

[0424] Hardware: Smartphones, servers

[0425] Software: Mobile applications, emotion engines, generative AI models

[0426] Program Processing

[0427] 1. Obtaining and displaying market prices

[0428] The server retrieves the market price of the item from the database by executing a SQL query like "SELECT FROM market_prices WHERE item_id = 'item123'".

[0429] The server sends the obtained market price to the mobile application on the smartphone. Specifically, the server uses the REST API to send a request such as "GET / api / market_prices / item123".

[0430] The terminal (smartphone) displays the received market price on the screen. Specifically, the terminal executes code such as "item_price_text_view.setText('1000 yen')".

[0431] 2. Listing items

[0432] The user taps on the smartphone screen where the market price is displayed. Specifically, the user taps on the part of the smartphone case where the market price is displayed.

[0433] The device displays the listing screen for the tapped item. Specifically, the device executes code such as "startActivity(new Intent(this, ListingActivity.class))".

[0434] The user inputs the listing price and listing comment on the listing screen and presses the listing button. Specifically, the user inputs "Listing price: 1000 yen" and "Listing comment: Brand new, unused" and presses the "List" button.

[0435] The terminal sends the entered information to the server. Specifically, the terminal sends a request such as "POST / api / listings" and includes the listing information in the body.

[0436] The server stores the received information in a database and lists the item. Specifically, the server executes a SQL query like "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'item123')".

[0437] 3. Listing recommendations based on emotion engine

[0438] The emotion engine analyzes the user's emotion data. Specifically, the emotion engine executes a function such as "analyzeEmotion(user_data)".

[0439] When a user expresses the emotion of joy, the emotion engine recommends listing items that the user feels joy in. Specifically, the emotion engine executes a function such as "recommendItem('joy')" to determine the recommended items.

[0440] If a user expresses an emotion of sadness, the emotion engine recommends listing items that make the user feel sad. Specifically, the emotion engine executes a function such as "recommendItem('sadness')" to determine the recommended items.

[0441] The device notifies the user of the recommendations from the emotion engine. Specifically, the device executes a code such as "showNotification('There are recommended items for sale')".

[0442] Examples

[0443] Obtaining and displaying market prices

[0444] The server executes the SQL query "SELECT FROM market_prices WHERE item_id = 'smartphone_case'" and obtains the market price of "1,000 yen."

[0445] The server sends a "GET / api / market_prices / smartphone_case" request to the smartphone.

[0446] The terminal executes the code "item_price_text_view.setText('1000 yen')" and displays "1000 yen" on the screen.

[0447] Selling items

[0448] When the user taps on the market price for "Smartphone Case", the device executes the "startActivity(new Intent(this, ListingActivity.class))" code and displays the listing screen.

[0449] The user enters "Listing price: 1,000 yen" and "Listing comment: Brand new, unused" and presses the "List" button.

[0450] The terminal sends a "POST / api / listings" request to the server and includes the listing information in the body.

[0451] The server executes the SQL query "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'smartphone_case')" to complete the listing.

[0452] Listing recommendations based on emotion engine

[0453] The emotion engine executes the function “analyzeEmotion(user_data)” and determines that the user is expressing the emotion of happiness.

[0454] The emotion engine executes the function "recommendItem('joy')" and determines that "smartphone case" is the recommended item.

[0455] The terminal executes the code "showNotification('There are recommended items for sale')" to notify the user.

[0456] Examples of prompt statements

[0457] "When a user expresses the emotion of joy, what items do you recommend them to sell?"

[0458] "When a user expresses sadness, what items do you recommend they list?"

[0459] In this way, the system helps users list items based on their sentiment and market prices.

[0460] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0461] Step 1:

[0462] Get market prices

[0463] The server retrieves the market price of the item from the database.

[0464] Input: Thing ID (e.g. 'item123')

[0465] Data processing: The server executes the SQL query "SELECT FROM market_prices WHERE item_id = 'item123'".

[0466] Output: Market price data (e.g. '1000 yen')

[0467] Step 2:

[0468] Sending quotes

[0469] The server transmits the obtained market price to a mobile application on the smartphone.

[0470] Input: Market price data (e.g. '1000 yen')

[0471] Data processing: The server uses the REST API to send a request for “GET / api / market_prices / item123”.

[0472] Output: Quote price data is sent to your smartphone.

[0473] Step 3:

[0474] Display of market price

[0475] The terminal (smartphone) displays the received market price on the screen.

[0476] Input: Market price data (e.g. '1000 yen')

[0477] Data processing: The terminal executes the code "item_price_text_view.setText('1000 yen')".

[0478] Output: The market price is displayed on the smartphone screen.

[0479] Step 4:

[0480] Selection of items

[0481] The user taps on the item whose market price is displayed on the smartphone screen.

[0482] Input: User taps

[0483] Data processing: The device obtains the ID of the tapped object.

[0484] Output: The ID of the item (e.g. 'item123')

[0485] Step 5:

[0486] Display of listing screen

[0487] The device displays the listing screen for the item that was tapped.

[0488] Input: Thing ID (e.g. 'item123')

[0489] Data processing: The device executes the code "startActivity(new Intent(this, ListingActivity.class))".

[0490] Output: The listing screen is displayed.

[0491] Step 6:

[0492] Entering listing information

[0493] The user inputs the listing price and listing comment on the listing screen and presses the listing button.

[0494] Input: Listing price (e.g. '1000 yen'), listing comment (e.g. 'Brand new, never used')

[0495] Data processing: The terminal retrieves the entered information.

[0496] Output: Listing information (e.g. '1000 yen', 'Brand new, never used')

[0497] Step 7:

[0498] Submit your listing

[0499] The terminal transmits the input information to the server.

[0500] Input: Listing information (e.g. '1000 yen', 'Brand new, never used')

[0501] Data processing: The terminal sends a “POST / api / listings” request and includes the listing information in the body.

[0502] Output: The listing is sent to the server.

[0503] Step 8:

[0504] Completing the listing

[0505] The server stores the received information in a database and lists the item.

[0506] Input: Listing information (e.g. '1000 yen', 'Brand new, never used')

[0507] Data processing: The server executes the SQL query "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'item123')".

[0508] Output: The listing is completed and saved to the database.

[0509] Step 9:

[0510] Emotion data analysis

[0511] The emotion engine analyzes the user's emotion data.

[0512] Input: User emotion data (e.g. 'joy')

[0513] Data processing: The emotion engine executes the function “analyzeEmotion(user_data)”.

[0514] Output: Sentiment analysis result (e.g. 'joy')

[0515] Step 10:

[0516] Decision on recommended products

[0517] When a user expresses the emotion of joy, the emotion engine recommends listing items that bring joy to the user.

[0518] Input: Sentiment analysis result (e.g. 'joy')

[0519] Data processing: The emotion engine executes the function “recommendItem('joy')” to determine the recommended items.

[0520] Output: Recommended item (e.g. 'phone case')

[0521] Step 11:

[0522] Recommendation Notification

[0523] The terminal notifies the user of the recommendations from the emotion engine.

[0524] Input: Recommended item (e.g. 'phone case')

[0525] Data processing: The terminal executes the code "showNotification('There are recommended items for sale')".

[0526] Output: A notification is displayed to the user.

[0527] (Application example 2)

[0528] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0529] In conventional secondary distribution systems, it was difficult for users to grasp the market price when listing items, and listing recommendations were not based on users' emotions. This made it difficult for users to list items at the appropriate price, and did not increase their motivation to list items.

[0530] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0531] In this invention, the server includes image analysis means, means for calculating a market price in secondary distribution of an item based on image data obtained by the image analysis means, means for displaying the calculated market price in augmented reality, means for listing an item based on the displayed market price, emotion recognition means for recognizing an emotion of a user, and means for recommending listing based on the recognized emotion. This enables a user to list an item at an appropriate price, and furthermore, since listing recommendations are made based on the emotion of the user, it is possible to increase the motivation to list an item.

[0532] An "image analysis means" is a device or software that analyzes image data acquired through a camera and recognizes the characteristics and condition of an object.

[0533] The "market price calculation means" refers to a device or software for calculating the price of an item in the secondary market based on the data obtained by the image analysis means.

[0534] "Means for displaying in augmented reality" refers to a device or software for displaying the calculated market price overlaid on a real-world image.

[0535] A "means for listing" is a device or software that allows a user to list an item based on the displayed market price.

[0536] An "emotion recognition means" is a device or software for analyzing and recognizing emotions from a user's facial expressions, voice, etc.

[0537] The "means for recommending items" is a device or software for suggesting appropriate items for sale based on the recognized user's sentiment.

[0538] The following system configuration will be described as an embodiment of the present invention.

[0539] First, the server is equipped with an image analysis means and analyzes image data acquired through the camera. This image analysis means includes a means for recognizing the characteristics and state of the object, and calculating the market price of the object in the secondary distribution market based on the data. The market price calculation means calculates the market price based on the acquired data, and transmits the result to the means for displaying in augmented reality.

[0540] Next, the terminal is provided with a means for displaying in augmented reality, and displays the calculated market price by superimposing it on the real image. Based on this display, the user can put the item up for sale. The means for listing provides an interface for listing the item based on the user's operation.

[0541] Furthermore, the server is equipped with an emotion recognition means for recognizing the emotion of the user. This emotion recognition means analyzes and recognizes the emotion of the user from facial expressions, voice, etc. Based on the recognized emotion, the listing recommendation means suggests listing of appropriate items.

[0542] As a specific example, when a user takes a picture of an item using a smartphone camera, the image data is sent to a server. The server analyzes the item using image analysis means and calculates a market price. The market price is displayed in augmented reality on the smartphone screen. When the user puts up an item for sale based on the price, emotion recognition means analyzes the user's emotions, and if the user is expressing an emotion of joy, for example, a message encouraging the listing is displayed.

[0543] The following hardware and software are used to realize this system:

[0544] Hardware: Smartphone camera

[0545] Software: OpenCV (image analysis library), EmotionRecognizer (emotion recognition library), market_data (market price data acquisition library), ecommerce_platform (mail order site API)

[0546] An example of a prompt sentence might be:

[0547] Develop an application that allows users to sell items using their smartphone camera. The application has the following features:

[0548] 1. Capture footage from the camera and recognize the user's emotions.

[0549] 2. Generate listing recommendations based on the perceived sentiment.

[0550] 3. Use the Market Data Acquisition library to obtain quotes for the selected instruments.

[0551] 4. The user is shown the listing screen and asked to enter the listing price and comments.

[0552] 5. Use the online shopping site's API to list your products.

[0553] In this way, users are able to list items at appropriate prices, and recommendations for listings are made based on the user's emotions, which increases the motivation to list items.

[0554] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0555] Step 1:

[0556] A user takes a picture of a product using the camera of a smartphone. The input is the video data from the camera, and the output is image data. This image data is sent to the server.

[0557] Step 2:

[0558] The server uses image analysis means to analyze the received image data. The input is image data, and the output is data indicating the characteristics and condition of the item. This data is sent to the market price calculation means.

[0559] Step 3:

[0560] The server uses a means for calculating a market price to calculate the market price of the item in the secondary market. The input is data indicating the characteristics and condition of the item, and the output is the market price. This market price is sent to the means for displaying it in augmented reality.

[0561] Step 4:

[0562] The terminal uses a means for displaying in augmented reality to display the calculated market price superimposed on a real image. The input is the market price, and the output is the market price displayed in augmented reality. The user confirms this display.

[0563] Step 5:

[0564] A user puts up an item for sale based on the market price. The input is the user's operation, and the output is the listing information. The listing information is sent to the server.

[0565] Step 6:

[0566] The server uses an emotion recognition means to analyze and recognize the emotion of the user. The input is the user's facial expression and voice data, and the output is the recognized emotion data. This emotion data is sent to the means for recommending items for sale.

[0567] Step 7:

[0568] The server generates a message recommending an item based on the recognized emotion. The input is the emotion data, and the output is the recommendation message. This message is sent to the terminal and displayed to the user.

[0569] Step 8:

[0570] The user checks the recommendation message and modifies the listing information if necessary. The input is the recommendation message and the user's operation, and the output is the final listing information. This listing information is sent to the server, and the product is listed through the API of the online shopping site.

[0571] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0572] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0573] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0574] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0575] [Second embodiment]

[0576] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0577] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0578] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0579] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0580] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0581] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0582] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0583] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0584] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0585] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0586] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0587] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0588] "Example 1"

[0589] In one embodiment of the present invention, a user takes a picture of an object they own using a camera such as a smartphone. The captured image is analyzed by an image analysis means to determine the type and condition of the object. Based on the results of this determination, the market price of the object in secondary distribution is calculated. The calculated market price is displayed to the user using AR (augmented reality) technology. Specifically, when the user looks at an object through the smartphone camera, the market price is displayed above the object.

[0590] "Example 2"

[0591] Furthermore, users can list items based on the displayed market price. Specifically, when a user taps on an item with a market price displayed on the smartphone screen, the listing screen for that item is displayed. On this screen, users can enter the listing price and listing comment, and then press the listing button to list the item.

[0592] The process flow of each embodiment will be described below.

[0593] "Example 1"

[0594] Step 1: The user takes a photo of an object they own using a smartphone or other camera.

[0595] Step 2: The captured image is analyzed by an image analysis means to determine the type and condition of the object.

[0596] Step 3: Based on the results of the assessment, the market price for the item in secondary distribution is calculated.

[0597] Step 4: The calculated price is displayed to the user using augmented reality (AR) technology. Specifically, when the user looks at an object through the smartphone camera, the price is displayed on top of the object.

[0598] "Example 2"

[0599] Step 1: The user taps on an object on their smartphone screen that displays a market price.

[0600] Step 2: When you tap, the listing screen for that item will appear.

[0601] Step 3: On this screen, the user enters the listing price and listing comments.

[0602] Step 4: You can list your item by pressing the list button.

[0603] Example 1

[0604] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0605] With conventional systems, it was difficult for users to quickly and accurately grasp the market price of the items they owned in the secondary distribution market. In addition, there was a lack of a way to display market price information in an easy-to-understand visual format, so users could not fully utilize it as a basis for making decisions when listing items. This made it difficult for users to buy and sell items at the appropriate price, resulting in a problem of reduced efficiency in transactions.

[0606] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0607] In this invention, the server includes a means for a user to photograph an object using a photographing device, a means for transmitting the photographed image to an analysis device, a means for the analysis device to determine the type and condition of the object using image analysis technology, a means for calculating the market price of the object in the secondary distribution market based on the determination result, a means for transmitting the calculated market price to a display device, and a means for the display device to display the market price using augmented reality technology. This allows the user to quickly and accurately grasp the market price of the object they own and to confirm it in a visually easy-to-understand manner. In addition, the user can list the object at an appropriate price, improving the efficiency of transactions.

[0608] A "user" is an individual or corporation that uses the system to photograph items, check market prices, and list them for sale.

[0609] An "imaging device" is a device such as a camera or smartphone that a user uses to photograph an object.

[0610] An "analysis device" is a computer system that receives captured images and uses image analysis technology to determine the type and condition of an object.

[0611] "Image analysis technology" refers to algorithms and software for analyzing image data to determine the type and condition of an object.

[0612] A "secondary market" is a market in which goods are bought and sold again after they have been in the hands of consumers.

[0613] "Market price" refers to the general trading price of a product in the secondary distribution market.

[0614] A "display device" is a device for visually displaying the calculated market price to a user, and includes a smartphone, tablet, etc.

[0615] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[0616] "Listing" refers to the act of a user offering something they own on the market for sale.

[0617] This invention is a system that quickly and accurately grasps the market price in the secondary distribution market of an item owned by a user and displays it in a visually easy-to-understand manner. A specific embodiment of this system is described below.

[0618] A user takes a picture of an object using a camera such as a smartphone. The captured image is sent to an analysis device via a smartphone application. The analysis device uses image analysis technology such as Google Cloud Vision API to determine the type and condition of the object in the image. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch can be determined.

[0619] The analysis device calculates the market price of the item in the secondary distribution market based on the judgment result. Specifically, it refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis result determines that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[0620] The calculated market price is sent from the analysis device to a smartphone. The smartphone displays the received market price using AR (Augmented Reality) technology. Specifically, when a user looks at an object through the smartphone's camera, the market price is displayed above the object. For example, when a user looks at a watch through the smartphone's camera, "Market price: \10,000" is displayed above the watch. Libraries such as ARKit (iOS) and ARCore (Android) are used for AR display.

[0621] This system allows users to quickly and accurately grasp the market price of their items and check it in a visually easy-to-understand format. It also allows users to list their items at an appropriate price, improving the efficiency of transactions.

[0622] As a concrete example, consider the case where a user has an old Seiko watch. The user activates the camera on their smartphone and takes a picture of the watch. The captured image is sent from the smartphone to a server, where the brand, model, and condition of the watch are analyzed using the Google Cloud Vision API. Based on the analysis results, a market price is calculated using the eBay API and past transaction prices of similar watches. The calculated market price is sent to the smartphone, and when the user looks at the watch through the camera, "Market Price: \10,000" is displayed above the watch.

[0623] An example of a prompt sentence might be:

[0624] "Write a program that takes a picture of an old Seiko watch, analyzes the brand, model and condition of the watch using the Google Cloud Vision API, calculates a market price by referencing the eBay API, and uses AR technology to display the market price on top of the watch through the smartphone camera."

[0625] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0626] Step 1:

[0627] The user takes a picture of an object using the smartphone camera.

[0628] Input: An image of an object taken by the user.

[0629] Specific operation: The user launches the smartphone camera app and takes a picture of an object they own. For example, if the user has an old watch, they can take a picture of the watch with the camera. When they press the capture button, the image is saved on the smartphone.

[0630] Output: The captured image data.

[0631] Step 2:

[0632] The terminal transmits the captured image to the server.

[0633] Input: The captured image data.

[0634] Specific operation: A smartphone application uploads image data to a server using an HTTP request. For example, the image data is encoded in Base64 format and sent in the body of a POST request.

[0635] Output: Image data sent to the server.

[0636] Step 3:

[0637] The server performs image analysis.

[0638] Input: Image data sent to the server.

[0639] How it works: The server uses image analysis technology such as the Google Cloud Vision API to determine the type and condition of objects in the image. The server sends image data to the API and receives the analysis results in return. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch are determined.

[0640] Output: Analysis results regarding the type and condition of the item.

[0641] Step 4:

[0642] The server calculates the market price of the item in the secondary distribution market.

[0643] Input: Analysis results regarding the type and condition of the item.

[0644] Specific operation: Based on the analysis results, the server refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis results determine that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[0645] Output: The calculated market price.

[0646] Step 5:

[0647] The server transmits the market price to the terminal.

[0648] Input: The calculated market price.

[0649] Specific operation: The server generates JSON data including the market price and returns it to the terminal as an HTTP response. For example, it returns data in the format {"item": "Seiko Watch", "price": "\10,000"}.

[0650] Output: Quote price data sent to the terminal.

[0651] Step 6:

[0652] The device displays market prices using AR technology.

[0653] Input: Quote price data sent to the terminal.

[0654] Specific operation: The smartphone uses AR (Augmented Reality) technology to display the received market price. Specifically, the market price is overlaid on the user's smartphone camera view. For example, when the user looks at their watch through the camera, "Market price: ¥10,000" is displayed on top of the watch. Libraries such as ARKit (iOS) and ARCore (Android) are used for the AR display.

[0655] Output: Quoted prices displayed using AR technology.

[0656] (Application example 1)

[0657] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0658] With conventional systems, it was difficult for users to easily grasp the market price of their items in the secondary market. In addition, in order to check the market price, users had to use multiple websites and applications, which was time-consuming. Furthermore, there was a lack of a way to check the market price in real time when appraising or purchasing items in a physical store.

[0659] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating the market price of the item in the secondary distribution based on the image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing the item based on the displayed market price, a means for photographing the item using the camera of the smart device and determining the type and condition of the item using image analysis technology, a means for acquiring market price data from a database based on the determination result and calculating the market price, a means for displaying the calculated market price on the screen of the smart device using AR technology, and a means for generating a prompt sentence for calculating the market price using a generative AI model. This allows the user to easily check the market price of the item using the smart device and grasp the market price in real time when the item is appraised or purchased in a physical store.

[0660] "Image analysis means" is a technology for analyzing image data acquired through a camera and determining the type and condition of an object.

[0661] The "market price calculation means" is a technology for calculating the market price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[0662] "Means of displaying using AR (Augmented Reality)" refers to technology for overlaying the calculated market price on the screen of a smart device.

[0663] A "means of listing" refers to technology that allows an item to be listed on an online marketplace, etc., based on user operations.

[0664] "Smart device" is a general term for portable electronic devices such as smartphones and tablets that can connect to the Internet and have a camera function.

[0665] "Image analysis technology" is a technology for analyzing image data captured by a camera and extracting specific information.

[0666] "Database" means an information system that stores market data and other related information in an organized manner and enables it to be searched and retrieved as needed.

[0667] A “generative AI model” is a model that is trained to perform a specific task using artificial intelligence, such as in this case to generate prompts for calculating quotes.

[0668] A "prompt sentence" is an input sentence that causes a generative AI model to perform a specific task.

[0669] A system for implementing the present invention uses the following hardware and software.

[0670] Hardware used

[0671] Smart devices (smartphones, tablets, etc.)

[0672] Server (image analysis, database)

[0673] Software used

[0674] Image analysis library (e.g. OpenCV)

[0675] AR library (e.g. ARCore)

[0676] Database (e.g. MySQL)

[0677] Generative AI models (e.g. GPT-4)

[0678] System configuration and operation

[0679] 1. Image capture

[0680] The user uses the camera on their smart device to take a picture of the item they want to appraise, and the image data is temporarily stored on the smart device.

[0681] 2. Image Analysis

[0682] The smart device sends the captured image data to the server, which uses an image analysis library (e.g., OpenCV) to analyze the image data and determine the type and condition of the product.

[0683] 3. Market Calculation

[0684] Based on the results of image analysis, the server retrieves market price data for the relevant item in the secondary distribution market from a database (e.g. MySQL), and calculates the market price based on the retrieved data.

[0685] 4. AR display

[0686] The calculated market price is sent from the server to the smart device, which uses an AR library (e.g., ARCore) to display the market price overlaid on the product seen through the camera.

[0687] 5. Use of generative AI models

[0688] The server uses a generative AI model (e.g. GPT-4) to generate a prompt to calculate the market price. The generated prompt has the following format:

[0689] "What type of item is this, and what is the current going price on the secondary market?"

[0690] Examples

[0691] For example, consider the case where a user wants to purchase a used smartphone at a physical store. The user takes a picture of the product using the smartphone camera. The captured image is sent to the server and analyzed using OpenCV. Based on the analysis results, the market price of the same model in the secondary market is obtained from the database. The calculated market price is displayed on the smartphone screen using ARCore. In this way, the user can easily check the market price when purchasing the product at a physical store.

[0692] This system allows users to easily check the market price of items using their smart devices, and understand the real-time market price when having items appraised or purchased in a physical store.

[0693] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0694] Step 1:

[0695] The user uses the camera of the smart device to take a picture of the item to be appraised. The input is the product image, and the output is the image data temporarily stored in the smart device. In concrete terms, the user starts the camera app, frames the product, and presses the shutter button.

[0696] Step 2:

[0697] The terminal transmits the captured image data to the server. The input is the temporarily stored image data, and the output is the image data transmitted to the server. In concrete terms, the terminal uploads the image data to the server via an Internet connection.

[0698] Step 3:

[0699] The server uses an image analysis library (e.g. OpenCV) to analyze the image data and determine the type and condition of the product. The input is the image data sent to the server, and the output is data about the type and condition of the product. Specifically, the server runs an image analysis algorithm to extract features in the image and identify the product.

[0700] Step 4:

[0701] Based on the results of image analysis, the server retrieves secondary market price data for the relevant product from a database (e.g. MySQL). The input is data about the type and condition of the product, and the output is price data. Specifically, the server executes a database query to search for price information for the relevant product.

[0702] Step 5:

[0703] The server calculates the market price based on the acquired market price data. The input is the market price data, and the output is the calculated market price. Specifically, the server uses statistical methods and algorithms to analyze the market price data and calculate the average price and median price.

[0704] Step 6:

[0705] The server transmits the calculated market price to the smart device. The input is the calculated market price, and the output is the market price transmitted to the smart device. In particular, the server transmits market price data to the smart device via an internet connection.

[0706] Step 7:

[0707] The device uses an AR library (e.g. ARCore) to overlay the market price on the product seen through the camera. The input is the market price sent from the server, and the output is the market price displayed on the smart device screen. In concrete terms, the device overlays the market price on the camera image.

[0708] Step 8:

[0709] The server uses a generative AI model (e.g. GPT-4) to generate a prompt for calculating the market price. The input is data on the type and condition of the product, and the output is the generated prompt. In concrete terms, the server requests the generative AI model to generate a prompt, and obtains the following prompt:

[0710] "What type of item is this, and what is the current going price on the secondary market?"

[0711] In this way, users can easily check the market price of items using their smart devices and understand the market price in real time when having an item appraised or purchased in a physical store.

[0712] Example 2

[0713] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0714] With conventional systems, it was difficult to accurately calculate the market price of goods in the secondary distribution market and provide an environment where users could easily list items for sale. In addition, there was a lack of a way to effectively utilize past transaction data when calculating market prices, so it was not possible to provide users with reliable market price information. Furthermore, the listing process was complicated, placing a heavy burden on users.

[0715] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating a market price in the secondary distribution of an item based on image data obtained by the image analysis means, a means for calculating a market price using a generative AI model, a means for displaying the calculated market price, a means for listing an item based on the displayed market price, a means for inputting listing information based on a user's operation, a means for transmitting the input listing information to the server, and a means for storing the listing information received by the server in a database. This enables a user to easily list an item based on highly reliable market price information.

[0716] An "image analysis means" is a device or software that analyzes image data acquired through a camera and recognizes the characteristics and condition of an object.

[0717] The "market price calculation means" refers to a device or software for calculating the price of an item in the secondary market based on the data obtained by the image analysis means and past transaction data.

[0718] A "generative AI model" is a model that uses artificial intelligence techniques to analyze data and perform specific tasks, in this case to calculate quotes.

[0719] A "means for displaying quotes" is a device or software for visually presenting the calculated quotes to a user.

[0720] A "means for listing" is a device or software that supports a series of operations for a user to list an item on the secondary market.

[0721] "Means for inputting listing information" refers to a device or software that allows a user to input information such as the price and comments of the item being listed.

[0722] The "means for transmitting the selling information to the server" refers to a device or software for transmitting the selling information input by the user to the server.

[0723] The "means for storing in a database" refers to a device or software that stores the listing information received by the server in a database and makes it available for other users to view.

[0724] This invention is a system that allows a user to list items for sale using a smartphone, and includes an image analysis means, a means for calculating a market price, a generative AI model, a means for displaying a market price, a means for listing an item, a means for inputting listing information, a means for transmitting the listing information to a server, and a means for saving the listing information in a database.

[0725] The user starts the app on their smartphone and uses the camera to capture an image of the item they wish to sell. The image analysis means analyzes the captured image data and recognizes the characteristics and condition of the item. The analyzed data is then sent to the server.

[0726] The server retrieves historical trading data from a database (e.g., MySQL) and calculates quotes using a generative AI model (e.g., OpenAI's GPT-4). For example, it sends a prompt like the following to the generative AI model:

[0727] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[0728] Transaction 1: 400 yen

[0729] Transaction 2: 500 yen

[0730] Transaction 3: 600 yen

[0731] The generative AI model calculates the market price based on this data, and the server sends the calculated market price to the user's device.

[0732] The terminal displays the received market price on the screen. For example, it may say, "The market price for this book is 500 yen." The user selects the item to sell based on the displayed market price, and inputs the selling price and listing comment on the listing screen. For example, the user sets the selling price to 600 yen and inputs the comment, "The item is in good condition."

[0733] When the user presses the sell button, the terminal sends the entered selling information to the server, which then stores the received selling information in a database and makes it available for other users to view.

[0734] The system allows users to easily list items based on reliable market information, and by using generative AI models, it effectively leverages past transaction data to provide accurate market prices.

[0735] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0736] Step 1:

[0737] The user launches the app.

[0738] A user launches an app by tapping the app icon on the home screen of their smartphone. The input is the user's operation, and the output is the launch of the app.

[0739] Step 2:

[0740] A user uses a camera to capture an image of an object.

[0741] A user uses the camera function in the app to take a picture of the item they want to put up for sale. The input is the image data acquired through the camera, and the output is the image data sent to the image analysis means.

[0742] Step 3:

[0743] The server analyzes the image data.

[0744] The server uses image analysis means to analyze the acquired image data and recognize the features and state of the object. The input is the image data sent by the user, and the output is the analyzed feature data of the object.

[0745] Step 4:

[0746] The server calculates the market price.

[0747] The server retrieves historical trading data from the database and uses the generative AI model to calculate quotes. For example, it sends the following prompt to the generative AI model:

[0748] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[0749] Transaction 1: 400 yen

[0750] Transaction 2: 500 yen

[0751] Transaction 3: 600 yen

[0752] The input is historical trading data and a prompt, and the output is the calculated quote price.

[0753] Step 5:

[0754] The server transmits the market price to the terminal.

[0755] The server sends the calculated quotes to the user's terminal. The input is the calculated quote and the output is the quote sent to the terminal.

[0756] Step 6:

[0757] The terminal will display the market price.

[0758] The terminal displays the received market price on the screen. For example, it displays "The market price of this book is 500 yen." The input is the market price sent from the server, and the output is the market price displayed on the screen.

[0759] Step 7:

[0760] The user selects the item to sell.

[0761] The user taps the item they want to sell from the list of market prices displayed on the screen. The input is the user's operation, and the output is a detailed screen of the selected item.

[0762] Step 8:

[0763] The device displays the listing screen.

[0764] The terminal opens a listing screen that displays detailed information about the selected item. The input is the information about the item selected by the user, and the output is the listing screen.

[0765] Step 9:

[0766] The user enters listing information.

[0767] The user inputs the listing price and listing comment on the listing screen. For example, the listing price is set to 600 yen and the comment "Condition is good." The input is the listing information entered by the user, and the output is the entered listing information.

[0768] Step 10:

[0769] The user presses the sell button.

[0770] The user presses the "list" button to confirm the information they have entered. The input is the user's operation, and the output is the confirmed listing information.

[0771] Step 11:

[0772] The terminal transmits the listing information to the server.

[0773] The terminal transmits the inputted selling information to the server. The input is the confirmed selling information, and the output is the selling information transmitted to the server.

[0774] Step 12:

[0775] The server stores the listing information in a database.

[0776] The server stores the received listings in a database and makes them available for other users to view. The input is the listing sent to the server and the output is the listing stored in the database.

[0777] (Application example 2)

[0778] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0779] In the conventional secondary distribution of goods, it was difficult for users to grasp the appropriate market price, and the listing process was cumbersome. In addition, it was time-consuming for users to input the listing price and comments, and the listing information was not saved efficiently. This reduced user convenience and hindered the revitalization of the secondary distribution market.

[0780] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0781] In this invention, the server includes an image analysis means, a means for calculating a market price for an item in secondary distribution based on image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing an item based on the displayed market price, and a means for a user to input a listing price and a comment and store listing information. This allows a user to easily grasp an appropriate market price and efficiently list items. Furthermore, since the listing information is efficiently stored, user convenience is improved and the secondary distribution market is expected to be revitalized.

[0782] An "image analysis means" is a device or software that analyzes image data acquired through a camera and extracts features and attributes of objects.

[0783] The "market price calculation means" refers to a device or software that calculates the fair price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[0784] "Means of displaying using AR (Augmented Reality)" refers to a technology that displays the calculated market price by overlaying it on a real-world image, and is a device or software that allows users to visually confirm the market price.

[0785] "Means for listing" refers to a device or software that provides an interface for users to list items based on market prices and allows for the input and storage of listing information.

[0786] The "means for inputting the listing price and comments and storing the listing information" refers to a device or software that allows the user to input the listing price and comments and stores the information in a database or the like.

[0787] A system for carrying out this invention has the following configuration. First, a user takes an image of an object using a smartphone. The smartphone is equipped with a camera, and image data acquired through this camera is sent to image analysis means. The image analysis means analyzes the image data and extracts features and attributes of the object.

[0788] Next, based on the data obtained by the image analysis means, the server calculates the fair price of the item in the secondary distribution market. This is the means of calculating the market price. The calculated market price is displayed on the smartphone display using AR (Augmented Reality) technology. This allows the user to visually check the market price by overlaying it on an image of the real world.

[0789] After the user checks the market price, an interface for entering the listing price and comments will be displayed on the smartphone screen. The user enters the listing price and comments through this interface and saves the listing information by pressing the listing button. This is the means for listing and saving the listing information.

[0790] The hardware used is a smartphone, and the software uses Flask (a Python web framework). Data processing and calculation involve obtaining market prices and storing listing information. Specifically, computer vision technology is used for image analysis, and machine learning models are used to calculate market prices.

[0791] As a concrete example, the prompt text when a user lists "item1" is as follows:

[0792] Example prompt:

[0793] A user is trying to list "item1". Get the market price, set the listing price to 1500 yen, and set the comment to "Brand new, unused."

[0794] In this way, users can easily check market prices and efficiently carry out the listing process. Furthermore, because listing information is efficiently stored, user convenience is improved and the secondary distribution market is expected to become more active.

[0795] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0796] Step 1:

[0797] A user takes an image of an object using the smartphone camera.

[0798] Input: Image of thing

[0799] Output: Image data

[0800] Specific operation: A user starts the camera app on their smartphone and takes a picture of an object. The captured image data is saved in the smartphone.

[0801] Step 2:

[0802] The terminal uses image analysis means to analyze the captured image data.

[0803] Input: Image data

[0804] Output: Feature and attribute data of objects

[0805] How it works: Image analysis software in the smartphone processes image data and extracts the object's features (e.g., shape, color, size) and attributes (e.g., brand, condition).

[0806] Step 3:

[0807] The server calculates the market price of the item in the secondary distribution market based on the data obtained by the image analysis means.

[0808] Input: Features and attribute data of objects

[0809] Output: Market price data

[0810] Specific operation: The server uses a machine learning model to input the item's characteristics and attributes, and calculates the market price based on past transaction data and market data.

[0811] Step 4:

[0812] The device displays the calculated market price using AR (augmented reality) technology.

[0813] Input: Market price data

[0814] Output: AR displayed market price

[0815] How it works: The smartphone's AR app receives market price data and displays the market price by overlaying it on real-world images via the camera.

[0816] Step 5:

[0817] The user enters the listing price and comments on the smartphone screen.

[0818] Input: Market price data, user-entered listing prices and comments

[0819] Output: Listing information

[0820] Specific operation: The user enters the listing price and comments into the listing interface on the smartphone and presses the listing button. The entered information is compiled into the listing information.

[0821] Step 6:

[0822] The server stores the listing information.

[0823] Input: Listing information

[0824] Output: Saved listings

[0825] Specific operation: The server saves the listing information to the database and notifies the user that the listing is complete.

[0826] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0827] "Example 1"

[0828] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine recognizes emotions from the user's facial expressions and tone of voice. Specifically, the user takes a picture of his or her face through the smartphone camera, and recognizes emotions such as joy, sadness, surprise, and anger from the facial expressions. In addition, the user records his or her voice through the smartphone microphone, and recognizes emotions from the tone of the voice.

[0829] "Example 2"

[0830] Furthermore, the emotion engine recommends listing items based on the user's emotions. Specifically, if a user expresses an emotion of happiness, it recommends listing items that make the user happy. Conversely, if a user expresses an emotion of sadness, it recommends listing items that make the user sad. In this way, the emotion engine understands the user's emotions and helps the user list items based on them.

[0831] The process flow of each embodiment will be described below.

[0832] "Example 1"

[0833] Step 1: The user takes a picture of their face using their smartphone camera.

[0834] Step 2: The emotion engine recognizes emotions such as happiness, sadness, surprise, and anger from the captured facial expressions.

[0835] Step 3: The user records their voice through the smartphone's microphone.

[0836] Step 4: The emotion engine identifies emotions from the tone of the recorded voice.

[0837] "Example 2"

[0838] Step 1: The emotion engine recognizes the user's emotion.

[0839] Step 2: If the user expresses the emotion of joy, the emotion engine will recommend listing something that brings the user joy.

[0840] Step 3: If the user indicates the emotion of sadness, the emotion engine will recommend that the user list something that makes them sad.

[0841] Example 1

[0842] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0843] In the conventional system, it was difficult for users to easily grasp the secondary market price of the items they owned, and the information provided did not take into account the users' feelings. As a result, it was difficult for users to make decisions when listing items, and there was an issue that the user experience did not improve.

[0844] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0845] In this invention, the server includes an image analysis means, a means for calculating the secondary market price of the item, a means for displaying the market price in augmented reality, a means for recognizing the user's emotions from their facial expressions and tone of voice, and a means for providing information to the user based on the recognized emotions. This allows the user to easily grasp the secondary market price of the item they own, and to receive appropriate information according to their emotions.

[0846] The "image analysis means" is a means for analyzing image data acquired through a camera and determining the type and condition of an object.

[0847] The "means for calculating the market price in the secondary distribution of goods" refers to a means for calculating the trading price in the secondary distribution market of goods based on the data obtained by the image analysis means.

[0848] "Means for displaying using augmented reality" refers to means for visually displaying the calculated market price to the user using augmented reality technology.

[0849] "Means for recognizing emotions from a user's facial expressions and tone of voice" refers to means for analyzing a user's facial expressions and tone of voice to identify emotions.

[0850] The "means for providing information to a user based on a recognized emotion" refers to a means for providing appropriate information to a user in accordance with the emotion of the user identified by the emotion recognition means.

[0851] A "means for listing items" is a means for a user to list items they own on the secondary market based on the displayed market price.

[0852] The present invention is a system that allows a user to easily grasp the secondary market price of an item owned by the user and provides information according to the user's emotions. A specific embodiment of this system will be described below.

[0853] System Configuration

[0854] This system consists of a user's terminal (smartphone), a server, an image analysis means, a market price calculation means, an augmented reality display means, an emotion recognition means, an information provision means, and an listing means.

[0855] Hardware and Software

[0856] Device (smartphone): Camera, microphone, AR app (e.g. ARKit, ARCore)

[0857] Server: Image analysis software (e.g. Google Cloud Vision API), market price calculation software (e.g. eBay API, Amazon API), emotion recognition software (e.g. Microsoft Azure Emotion API)

[0858] Processing flow

[0859] 1. The user takes a photo of an object

[0860] Users use the smartphone camera to take a picture of an object they own. For example, if a user wants to take a picture of an old book, they start the smartphone camera app and take a picture of the book cover.

[0861] 2. The device sends the image to the server

[0862] The device sends the captured image to the server. At this time, the device uses an Internet connection to upload the image data to the server.

[0863] 3. The server analyzes the image

[0864] The server analyzes the received images. Specifically, the server uses the Google Cloud Vision API to determine the type and condition of the objects in the image. For example, the server can identify the title, author, and condition of a book (new, used, damaged, etc.).

[0865] 4. The server calculates the secondary market price of the item

[0866] Based on the results of the search, the server calculates the market price of the item in the secondary market. To do this, the server refers to the eBay API or Amazon API to check the past transaction prices of similar items. For example, the server calculates the market price by averaging the transaction prices of the same book over the past year.

[0867] 5. The server sends the quotes to the terminal

[0868] The server then sends the calculated market price to the device, which receives the data and displays the market price using an AR app.

[0869] 6. The device displays market prices in AR

[0870] The device uses ARKit and ARCore to display a price quote on an object when the user looks at it through the smartphone camera. For example, when the user looks at a book through the camera, "Price quote: $1,500" is displayed on the book.

[0871] 7. Recognize user emotions

[0872] The user uses the smartphone camera to take a picture of their face and the smartphone microphone to record their voice, and the device transmits this data to a server.

[0873] 8. The server analyzes emotions

[0874] The server uses the Microsoft Azure Emotion API to identify emotions from the user's facial expressions. The server also recognizes emotions from the tone of the recorded voice. For example, the server recognizes a user's smile as "happiness" and a high-pitched voice as "excitement."

[0875] 9. The server sends the emotion recognition results to the device.

[0876] The server sends the emotion recognition results to the device, which receives the data and displays it to the user.

[0877] 10. The device displays the emotion recognition results.

[0878] The device will display the emotion recognition results to the user. For example, if the user is smiling, it will display "You are happy."

[0879] Examples

[0880] Example 1: Finding the market value of old books

[0881] 1. The user takes a photo of an old book with their smartphone camera.

[0882] 2. The device sends the image to the server.

[0883] 3. The server uses the Google Cloud Vision API to identify the book's title and status.

[0884] 4. The server calculates the market price by referencing the eBay API or Amazon API.

[0885] 5. The server sends the market price to the terminal.

[0886] 6. The device uses ARKit to display price quotes on top of the book.

[0887] Example 2: Emotion recognition

[0888] 1. The user takes a photo of their face with their smartphone camera and records their voice with the microphone.

[0889] 2. The device sends this data to the server.

[0890] 3. The server uses the Microsoft Azure Emotion API to identify emotions from facial expressions.

[0891] 4. The server recognizes emotions from the tone of the recorded voice.

[0892] 5. The server sends the emotion recognition results to the device.

[0893] 6. The device displays the emotion recognition results to the user.

[0894] Examples of prompt statements

[0895] "I want to find out the market price of an old book. I took a picture of the book with my smartphone camera. Please tell me the market price."

[0896] "Please recognize my emotions. I photographed my face with my phone camera and recorded my voice with a microphone."

[0897] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0898] Step 1:

[0899] The user takes a photo of an object

[0900] A user uses the smartphone camera to take a picture of an object they own. For example, if a user wants to take a picture of an old book, they start the smartphone camera app and take a picture of the book's cover. The input is the image of the object, and the output is the captured image data.

[0901] Step 2:

[0902] The device sends the image to the server.

[0903] The terminal sends the captured image to the server. At this time, the terminal uses an Internet connection to upload image data to the server. The input is the captured image data, and the output is the image data sent to the server.

[0904] Step 3:

[0905] The server analyzes the image

[0906] The server analyzes the images it receives. Specifically, the server uses image analysis software (e.g., Google Cloud Vision API) to determine the type and condition of the object in the image. For example, the server might identify the book's title, author, and condition (new, used, damaged, etc.). The input is the image data sent to the server, and the output is data about the type and condition of the object.

[0907] Step 4:

[0908] The server calculates the secondary market price of the item

[0909] Based on the results of the determination, the server calculates the market price of the item in the secondary market. To do this, the server refers to a database (e.g., eBay API or Amazon API) and investigates past transaction prices of similar items. For example, the server calculates the market price by averaging the transaction prices of the same book over the past year. The input is data about the type and condition of the item, and the output is the calculated market price.

[0910] Step 5:

[0911] The server sends the market price to the terminal.

[0912] The server sends the calculated market price to the terminal. The terminal receives this data and displays the market price using the AR app. The input is the calculated market price, and the output is the market price data sent to the terminal.

[0913] Step 6:

[0914] The device displays market prices in AR

[0915] The device uses ARKit and ARCore to display a current price on top of an object when the user looks at it through the smartphone camera. For example, when the user looks at a book through the camera, "Current Price: $1,500" is displayed on top of the book. The input is the current price data sent to the device, and the output is the current price displayed in AR.

[0916] Step 7:

[0917] Recognize user emotions

[0918] The user takes a picture of their face using the smartphone camera and records their voice using the smartphone microphone. The device sends these data to the server. The input is the user's face image data and voice recording data, and the output is emotion recognition data sent to the server.

[0919] Step 8:

[0920] The server analyzes emotions

[0921] The server uses emotion recognition software (e.g., Microsoft Azure Emotion API) to identify emotions from the user's facial expressions. The server also recognizes emotions from the tone of a recorded voice. For example, the server may recognize a user's smiling face as "happiness" and a high-pitched voice as "excitement." The input is the emotion recognition data sent to the server, and the output is the recognized emotion data.

[0922] Step 9:

[0923] The server sends the emotion recognition results to the device.

[0924] The server sends the emotion recognition results to the terminal. The terminal receives this data and displays it to the user. The input is the recognized emotion data, and the output is the emotion recognition result data sent to the terminal.

[0925] Step 10:

[0926] The device displays the emotion recognition results.

[0927] The terminal displays the emotion recognition result to the user. For example, if the user is smiling, "You are happy" is displayed. The input is the emotion recognition result data sent to the terminal, and the output is the emotion recognition result displayed to the user.

[0928] (Application example 1)

[0929] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0930] In the conventional secondary market, users had to spend a lot of time and effort to understand the market price of the items they own. In addition, there was also the issue that the user experience did not improve because information was provided uniformly without considering the user's feelings.

[0931] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0932] In this invention, the server includes an image analysis means, a means for calculating a market price in secondary distribution of an item based on image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing an item based on the displayed market price, an emotion recognition means for recognizing an emotion from a user's facial expression and tone of voice, and a means for providing feedback based on the recognized emotion. This allows a user to quickly and easily grasp the market price of an item that he or she owns, and to receive appropriate feedback according to the emotion.

[0933] The "image analysis means" is a means for analyzing image data acquired through a camera and determining the type and condition of an object.

[0934] The "market price calculation means" is a means for calculating the market price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[0935] "Means for displaying using AR (Augmented Reality)" refers to a means for displaying the calculated market price by overlaying it on a real image.

[0936] A "means for listing items" is a means for a user to list items on the secondary market based on the displayed market price.

[0937] An "emotion recognition means" is a means for analyzing a user's facial expressions and tone of voice to recognize the user's emotions.

[0938] The "means for providing feedback" is a means for providing appropriate feedback to a user based on the recognized user sentiment.

[0939] A system for implementing this invention has the following configuration. First, a user takes a picture of an item they own using the smartphone camera. The captured image is analyzed by image analysis means to determine the type and condition of the item. Next, the market price of the item in the secondary distribution market is calculated based on the results of the determination. The calculated market price is displayed to the user using AR (Augmented Reality) technology. Specifically, when the user looks at an item through the smartphone camera, the market price is displayed above the item.

[0940] Furthermore, an emotion recognition means is incorporated, and emotions are recognized from the user's facial expressions and tone of voice. Specifically, the user takes a picture of his / her face using the smartphone camera, and emotions such as joy, sadness, surprise, and anger are recognized from the facial expressions. In addition, the user records his / her voice using the smartphone microphone, and emotions are recognized from the tone of the voice. Based on the recognized emotions, the feedback providing means provides appropriate feedback to the user.

[0941] Hardware and Software Used:

[0942] Hardware: Smartphone (camera, microphone)

[0943] Software: OpenCV (image processing library), Keras (deep learning library for image analysis and emotion recognition), pyttsx3 (voice synthesis library), dlib (face detection library)

[0944] Data processing and calculation:

[0945] 1. Image analysis: Images taken with a smartphone camera are read using OpenCV, and the type and condition of the item are determined using a Keras model, and the market price in the secondary market is calculated.

[0946] 2. Emotion recognition: Face detection is performed using dlib on images of the user's face taken with a smartphone camera, and emotions are recognized using a Keras model.

[0947] 3. AR display: Using OpenCV, the calculated market price is displayed on the image.

[0948] 4. Audio feedback: Use pyttsx3 to provide audio feedback according to the recognized emotion.

[0949] Examples:

[0950] When users take a picture of their old phone with their smartphone camera, the phone's market value on the secondary market is displayed on the screen, and if the user smiles, voice feedback is provided saying, "This item has good resale value."

[0951] Example prompt for a generative AI model:

[0952] Analyze the images taken by the user using the smartphone camera and calculate the market price of the item in the secondary market. Also, recognize the emotions of the user from their facial expressions and tone of voice and provide feedback according to their emotions.

[0953] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0954] Step 1:

[0955] A user takes a picture of an object they own using the camera of their smartphone. The input is the image data acquired through the camera, and the output is that image data. In concrete terms, the user starts the camera app on their smartphone, puts the object in the frame, and presses the capture button.

[0956] Step 2:

[0957] The terminal transmits the captured image data to the image analysis means. The input is the image data acquired in step 1, and the output is the data transmitted to the image analysis means. In concrete terms, the smartphone application transmits the image data to the server.

[0958] Step 3:

[0959] The server uses image analysis means to analyze the image data and determine the type and status of the object. The input is the image data sent in step 2, and the output is data related to the type and status of the object. In concrete terms, an image analysis model (using Keras) on the server processes the image data and determines the type and status of the object.

[0960] Step 4:

[0961] The server calculates the market price of the item in the secondary market based on the judgment result. The input is the data on the type and condition of the item obtained in step 3, and the output is the market price. In concrete terms, the server refers to the database and calculates the market price based on the market prices of similar items.

[0962] Step 5:

[0963] The server sends the calculated market price to the terminal, which then displays the market price using AR technology. The input is the market price calculated in step 4, and the output is the market price displayed on the smartphone screen. In concrete terms, the smartphone application overlays the market price on the camera image.

[0964] Step 6:

[0965] A user takes a picture of his / her face through the smartphone camera, and the terminal transmits the image data to the emotion recognition means. The input is the face image data acquired through the camera, and the output is the data transmitted to the emotion recognition means. In concrete terms, the user points the smartphone camera at himself / herself and takes a picture of his / her face.

[0966] Step 7:

[0967] The server uses emotion recognition means to recognize emotions from the user's facial expressions and tone of voice. The input is the facial image data and voice data sent in step 6, and the output is the recognized emotion data. In concrete terms, the emotion recognition model (using Keras) on the server analyzes the facial image and voice data to determine the emotion.

[0968] Step 8:

[0969] The server generates feedback based on the recognized emotion and sends it to the terminal. The input is the emotion data obtained in step 7, and the output is a feedback message. In concrete terms, the server generates a feedback message according to the emotion data and sends it to the terminal.

[0970] Step 9:

[0971] The terminal provides the feedback message to the user using a speech synthesis engine. The input is the feedback message sent in step 8, and the output is the voice feedback provided to the user. In concrete terms, the smartphone application uses pyttsx3 to play back the feedback message aloud.

[0972] Example 2

[0973] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0974] In conventional secondary distribution systems, it was difficult for users to grasp the appropriate market price, and listing recommendations were not made based on the user's emotions. This made it difficult for users to list items at the optimal time, which resulted in the revitalization of the secondary distribution market being hindered. Furthermore, because listing recommendations were not made based on the user's emotions, there was a possibility that user satisfaction would decrease.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0976] In this invention, the server includes an image analysis means, a market price calculation means, a market price display means, a listing means, an emotion analysis means, and a listing recommendation means. This allows the user to understand the appropriate market price and receive listing recommendations based on emotions. This allows the user to list items at the optimal time, promoting the revitalization of the secondary distribution market. Furthermore, listing recommendations that take the user's emotions into consideration improve user satisfaction.

[0977] The "image analysis means" refers to a means for analyzing image data acquired using a camera or other image acquisition device, and extracting necessary information.

[0978] The "market price calculation means" is a means for calculating the market price in the secondary distribution market of an item based on the data obtained by the image analysis means.

[0979] The "market price display means" is a means for visually displaying the calculated market price to the user.

[0980] A "listing means" is a means by which a user lists an item on the secondary distribution market.

[0981] The "emotion analysis means" is a means for analyzing the user's emotional data and determining the user's emotional state.

[0982] The "item recommendation means" is a means for recommending appropriate items to a user for sale based on the user's emotion data analyzed by the emotion analysis means.

[0983] This invention is a system that allows a user to grasp an appropriate market price when putting an item up for sale on a secondary market, and further allows the user to receive a recommendation for listing based on the user's emotions. This system includes an image analysis means, a market price calculation means, a market price display means, a listing means, an emotion analysis means, and a listing recommendation means.

[0984] Hardware and Software Used

[0985] Hardware: Smartphones, servers

[0986] Software: Mobile applications, emotion engines, generative AI models

[0987] Program Processing

[0988] 1. Obtaining and displaying market prices

[0989] The server retrieves the market price of the item from the database by executing a SQL query like "SELECT FROM market_prices WHERE item_id = 'item123'".

[0990] The server sends the obtained market price to the mobile application on the smartphone. Specifically, the server uses the REST API to send a request such as "GET / api / market_prices / item123".

[0991] The terminal (smartphone) displays the received market price on the screen. Specifically, the terminal executes code such as "item_price_text_view.setText('1000 yen')".

[0992] 2. Listing items

[0993] The user taps on the smartphone screen where the market price is displayed. Specifically, the user taps on the part of the smartphone case where the market price is displayed.

[0994] The device displays the listing screen for the tapped item. Specifically, the device executes code such as "startActivity(new Intent(this, ListingActivity.class))".

[0995] The user inputs the listing price and listing comment on the listing screen and presses the listing button. Specifically, the user inputs "Listing price: 1000 yen" and "Listing comment: Brand new, unused" and presses the "List" button.

[0996] The terminal sends the entered information to the server. Specifically, the terminal sends a request such as "POST / api / listings" and includes the listing information in the body.

[0997] The server stores the received information in a database and lists the item. Specifically, the server executes a SQL query like "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'item123')".

[0998] 3. Listing recommendations based on emotion engine

[0999] The emotion engine analyzes the user's emotion data. Specifically, the emotion engine executes a function such as "analyzeEmotion(user_data)".

[1000] When a user expresses the emotion of joy, the emotion engine recommends listing items that the user feels joy in. Specifically, the emotion engine executes a function such as "recommendItem('joy')" to determine the recommended items.

[1001] If a user expresses an emotion of sadness, the emotion engine recommends listing items that make the user feel sad. Specifically, the emotion engine executes a function such as "recommendItem('sadness')" to determine the recommended items.

[1002] The device notifies the user of the recommendations from the emotion engine. Specifically, the device executes a code such as "showNotification('There are recommended items for sale')".

[1003] Examples

[1004] Obtaining and displaying market prices

[1005] The server executes the SQL query "SELECT FROM market_prices WHERE item_id = 'smartphone_case'" and obtains the market price of "1,000 yen."

[1006] The server sends a "GET / api / market_prices / smartphone_case" request to the smartphone.

[1007] The terminal executes the code "item_price_text_view.setText('1000 yen')" and displays "1000 yen" on the screen.

[1008] Selling items

[1009] When the user taps on the market price for "Smartphone Case", the device executes the "startActivity(new Intent(this, ListingActivity.class))" code and displays the listing screen.

[1010] The user enters "Listing price: 1,000 yen" and "Listing comment: Brand new, unused" and presses the "List" button.

[1011] The terminal sends a "POST / api / listings" request to the server and includes the listing information in the body.

[1012] The server executes the SQL query "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'smartphone_case')" to complete the listing.

[1013] Listing recommendations based on emotion engine

[1014] The emotion engine executes the function “analyzeEmotion(user_data)” and determines that the user is expressing the emotion of happiness.

[1015] The emotion engine executes the function "recommendItem('joy')" and determines that "smartphone case" is the recommended item.

[1016] The terminal executes the code "showNotification('There are recommended items for sale')" to notify the user.

[1017] Examples of prompt statements

[1018] "When a user expresses the emotion of joy, what items do you recommend them to sell?"

[1019] "When a user expresses sadness, what items do you recommend they list?"

[1020] In this way, the system helps users list items based on their sentiment and market prices.

[1021] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1022] Step 1:

[1023] Get market prices

[1024] The server retrieves the market price of the item from the database.

[1025] Input: Thing ID (e.g. 'item123')

[1026] Data processing: The server executes the SQL query "SELECT FROM market_prices WHERE item_id = 'item123'".

[1027] Output: Market price data (e.g. '1000 yen')

[1028] Step 2:

[1029] Sending quotes

[1030] The server transmits the obtained market price to a mobile application on the smartphone.

[1031] Input: Market price data (e.g. '1000 yen')

[1032] Data processing: The server uses the REST API to send a request for “GET / api / market_prices / item123”.

[1033] Output: Quote price data is sent to your smartphone.

[1034] Step 3:

[1035] Display of market price

[1036] The terminal (smartphone) displays the received market price on the screen.

[1037] Input: Market price data (e.g. '1000 yen')

[1038] Data processing: The terminal executes the code "item_price_text_view.setText('1000 yen')".

[1039] Output: The market price is displayed on the smartphone screen.

[1040] Step 4:

[1041] Selection of items

[1042] The user taps on the item whose market price is displayed on the smartphone screen.

[1043] Input: User taps

[1044] Data processing: The device obtains the ID of the tapped object.

[1045] Output: The ID of the item (e.g. 'item123')

[1046] Step 5:

[1047] Display of listing screen

[1048] The device displays the listing screen for the item that was tapped.

[1049] Input: Thing ID (e.g. 'item123')

[1050] Data processing: The device executes the code "startActivity(new Intent(this, ListingActivity.class))".

[1051] Output: The listing screen is displayed.

[1052] Step 6:

[1053] Entering listing information

[1054] The user inputs the listing price and listing comment on the listing screen and presses the listing button.

[1055] Input: Listing price (e.g. '1000 yen'), listing comment (e.g. 'Brand new, never used')

[1056] Data processing: The terminal retrieves the entered information.

[1057] Output: Listing information (e.g. '1000 yen', 'Brand new, never used')

[1058] Step 7:

[1059] Submit your listing

[1060] The terminal transmits the input information to the server.

[1061] Input: Listing information (e.g. '1000 yen', 'Brand new, never used')

[1062] Data processing: The terminal sends a “POST / api / listings” request and includes the listing information in the body.

[1063] Output: The listing is sent to the server.

[1064] Step 8:

[1065] Completing the listing

[1066] The server stores the received information in a database and lists the item.

[1067] Input: Listing information (e.g. '1000 yen', 'Brand new, never used')

[1068] Data processing: The server executes the SQL query "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'item123')".

[1069] Output: The listing is completed and saved to the database.

[1070] Step 9:

[1071] Emotion data analysis

[1072] The emotion engine analyzes the user's emotion data.

[1073] Input: User emotion data (e.g. 'joy')

[1074] Data processing: The emotion engine executes the function “analyzeEmotion(user_data)”.

[1075] Output: Sentiment analysis result (e.g. 'joy')

[1076] Step 10:

[1077] Decision on recommended products

[1078] When a user expresses the emotion of joy, the emotion engine recommends listing items that bring joy to the user.

[1079] Input: Sentiment analysis result (e.g. 'joy')

[1080] Data processing: The emotion engine executes the function “recommendItem('joy')” to determine the recommended items.

[1081] Output: Recommended item (e.g. 'phone case')

[1082] Step 11:

[1083] Recommendation Notification

[1084] The terminal notifies the user of the recommendations from the emotion engine.

[1085] Input: Recommended item (e.g. 'phone case')

[1086] Data processing: The terminal executes the code "showNotification('There are recommended items for sale')".

[1087] Output: A notification is displayed to the user.

[1088] (Application example 2)

[1089] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1090] In conventional secondary distribution systems, it was difficult for users to grasp the market price when listing items, and listing recommendations were not based on the user's feelings. This made it difficult for users to list items at the appropriate price, and did not increase their motivation to list items.

[1091] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1092] In this invention, the server includes image analysis means, means for calculating a market price in secondary distribution of an item based on image data obtained by the image analysis means, means for displaying the calculated market price in augmented reality, means for listing an item based on the displayed market price, emotion recognition means for recognizing an emotion of a user, and means for recommending listing based on the recognized emotion. This enables a user to list an item at an appropriate price, and furthermore, since listing recommendations are made based on the emotion of the user, it is possible to increase the motivation to list an item.

[1093] An "image analysis means" is a device or software that analyzes image data acquired through a camera and recognizes the characteristics and condition of an object.

[1094] The "market price calculation means" refers to a device or software for calculating the price of an item in the secondary market based on the data obtained by the image analysis means.

[1095] "Means for displaying in augmented reality" refers to a device or software for displaying the calculated market price overlaid on a real-world image.

[1096] A "means for listing" is a device or software that allows a user to list an item based on the displayed market price.

[1097] An "emotion recognition means" is a device or software for analyzing and recognizing emotions from a user's facial expressions, voice, etc.

[1098] The "means for recommending items" is a device or software for suggesting appropriate items for sale based on the recognized user's sentiment.

[1099] The following system configuration will be described as an embodiment of the present invention.

[1100] First, the server is equipped with an image analysis means and analyzes image data acquired through the camera. This image analysis means includes a means for recognizing the characteristics and state of the object, and calculating the market price of the object in the secondary distribution market based on the data. The market price calculation means calculates the market price based on the acquired data, and transmits the result to the means for displaying in augmented reality.

[1101] Next, the terminal is provided with a means for displaying in augmented reality, and displays the calculated market price by overlaying it on a real image. Based on this display, the user can put the item up for sale. The means for listing provides an interface for listing the item based on the user's operation.

[1102] Furthermore, the server is equipped with an emotion recognition means for recognizing the emotion of the user. This emotion recognition means analyzes and recognizes the emotion of the user from facial expressions, voice, etc. Based on the recognized emotion, the listing recommendation means suggests listing of appropriate items.

[1103] As a specific example, when a user takes a picture of an item using a smartphone camera, the image data is sent to a server. The server analyzes the item using image analysis means and calculates a market price. The market price is displayed in augmented reality on the smartphone screen. When the user puts up an item for sale based on the price, emotion recognition means analyzes the user's emotions, and if the user is expressing an emotion of joy, for example, a message encouraging the listing is displayed.

[1104] The following hardware and software are used to realize this system:

[1105] Hardware: Smartphone camera

[1106] Software: OpenCV (image analysis library), EmotionRecognizer (emotion recognition library), market_data (market price data acquisition library), ecommerce_platform (mail order site API)

[1107] An example of a prompt sentence might be:

[1108] Develop an application that allows users to sell items using their smartphone camera. The application has the following features:

[1109] 1. Capture footage from the camera and recognize the user's emotions.

[1110] 2. Generate listing recommendations based on the perceived sentiment.

[1111] 3. Use the Market Data Acquisition library to obtain quotes for the selected instruments.

[1112] 4. The user is shown the listing screen and asked to enter the listing price and comments.

[1113] 5. Use the online shopping site's API to list your products.

[1114] In this way, users are able to list items at appropriate prices, and recommendations for listings are made based on the user's emotions, which increases the motivation to list items.

[1115] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1116] Step 1:

[1117] A user takes a picture of a product using the camera of a smartphone. The input is the video data from the camera, and the output is image data. This image data is sent to the server.

[1118] Step 2:

[1119] The server uses image analysis means to analyze the received image data. The input is image data, and the output is data indicating the characteristics and condition of the item. This data is sent to the market price calculation means.

[1120] Step 3:

[1121] The server uses a means for calculating a market price to calculate the market price of the item in the secondary market. The input is data indicating the characteristics and condition of the item, and the output is the market price. This market price is sent to the means for displaying it in augmented reality.

[1122] Step 4:

[1123] The terminal uses a means for displaying in augmented reality to display the calculated market price superimposed on a real image. The input is the market price, and the output is the market price displayed in augmented reality. The user confirms this display.

[1124] Step 5:

[1125] A user puts up an item for sale based on the market price. The input is the user's operation, and the output is the listing information. The listing information is sent to the server.

[1126] Step 6:

[1127] The server uses an emotion recognition means to analyze and recognize the emotion of the user. The input is the user's facial expression and voice data, and the output is the recognized emotion data. This emotion data is sent to the means for recommending items for sale.

[1128] Step 7:

[1129] The server generates a message recommending an item based on the recognized emotion. The input is the emotion data, and the output is the recommendation message. This message is sent to the terminal and displayed to the user.

[1130] Step 8:

[1131] The user checks the recommendation message and modifies the listing information if necessary. The input is the recommendation message and the user's operation, and the output is the final listing information. This listing information is sent to the server, and the product is listed through the API of the online shopping site.

[1132] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1133] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1134] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1135] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1136] [Third embodiment]

[1137] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1138] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1140] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1141] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1142] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[1143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1144] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1145] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1146] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1147] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1148] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1149] "Example 1"

[1150] In one embodiment of the present invention, a user takes a picture of an object they own using a camera such as a smartphone. The captured image is analyzed by an image analysis means to determine the type and condition of the object. Based on the results of this determination, the market price of the object in secondary distribution is calculated. The calculated market price is displayed to the user using AR (augmented reality) technology. Specifically, when the user looks at an object through the smartphone camera, the market price is displayed above the object.

[1151] "Example 2"

[1152] Furthermore, users can list items based on the displayed market price. Specifically, when a user taps on an item with a market price displayed on the smartphone screen, the listing screen for that item is displayed. On this screen, users can enter the listing price and listing comment, and then press the listing button to list the item.

[1153] The process flow of each embodiment will be described below.

[1154] "Example 1"

[1155] Step 1: The user takes a photo of an object they own using a smartphone or other camera.

[1156] Step 2: The captured image is analyzed by an image analysis means to determine the type and condition of the object.

[1157] Step 3: Based on the results of the assessment, the market price for the item in secondary distribution is calculated.

[1158] Step 4: The calculated price is displayed to the user using augmented reality (AR) technology. Specifically, when the user looks at an object through the smartphone camera, the price is displayed on top of the object.

[1159] "Example 2"

[1160] Step 1: The user taps on an object on their smartphone screen that displays a market price.

[1161] Step 2: When you tap, the listing screen for that item will appear.

[1162] Step 3: On this screen, the user enters the listing price and listing comments.

[1163] Step 4: You can list your item by pressing the list button.

[1164] Example 1

[1165] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1166] With conventional systems, it was difficult for users to quickly and accurately grasp the market price of the items they owned in the secondary distribution market. In addition, there was a lack of a way to display market price information in an easy-to-understand visual format, so users could not fully utilize it as a basis for making decisions when listing items. This made it difficult for users to buy and sell items at the appropriate price, resulting in a problem of reduced efficiency in transactions.

[1167] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1168] In this invention, the server includes a means for a user to photograph an object using a photographing device, a means for transmitting the photographed image to an analysis device, a means for the analysis device to determine the type and condition of the object using image analysis technology, a means for calculating the market price of the object in the secondary distribution market based on the determination result, a means for transmitting the calculated market price to a display device, and a means for the display device to display the market price using augmented reality technology. This allows the user to quickly and accurately grasp the market price of the object they own and to confirm it in a visually easy-to-understand manner. In addition, the user can list the object at an appropriate price, improving the efficiency of transactions.

[1169] A "user" is an individual or corporation that uses the system to photograph items, check market prices, and list them for sale.

[1170] An "imaging device" is a device such as a camera or smartphone that a user uses to photograph an object.

[1171] An "analysis device" is a computer system that receives captured images and uses image analysis technology to determine the type and condition of an object.

[1172] "Image analysis technology" refers to algorithms and software for analyzing image data to determine the type and condition of an object.

[1173] A "secondary market" is a market in which goods are bought and sold again after they have been in the hands of consumers.

[1174] "Market price" refers to the general trading price of a product in the secondary distribution market.

[1175] A "display device" is a device for visually displaying the calculated market price to a user, and includes a smartphone, tablet, etc.

[1176] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[1177] "Listing" refers to the act of a user offering something they own on the market for sale.

[1178] This invention is a system that quickly and accurately grasps the market price in the secondary distribution market of an item owned by a user and displays it in a visually easy-to-understand manner. A specific embodiment of this system is described below.

[1179] A user takes a picture of an object using a camera such as a smartphone. The captured image is sent to an analysis device via a smartphone application. The analysis device uses image analysis technology such as Google Cloud Vision API to determine the type and condition of the object in the image. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch can be determined.

[1180] The analysis device calculates the market price of the item in the secondary distribution market based on the judgment result. Specifically, it refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis result determines that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[1181] The calculated market price is sent from the analysis device to a smartphone. The smartphone displays the received market price using AR (Augmented Reality) technology. Specifically, when a user looks at an object through the smartphone's camera, the market price is displayed above the object. For example, when a user looks at a watch through the smartphone's camera, "Market price: \10,000" is displayed above the watch. Libraries such as ARKit (iOS) and ARCore (Android) are used for AR display.

[1182] This system allows users to quickly and accurately grasp the market price of their items and check it in a visually easy-to-understand format. It also allows users to list their items at an appropriate price, improving the efficiency of transactions.

[1183] As a concrete example, consider the case where a user has an old Seiko watch. The user activates the camera on their smartphone and takes a picture of the watch. The captured image is sent from the smartphone to a server, where the brand, model, and condition of the watch are analyzed using the Google Cloud Vision API. Based on the analysis results, a market price is calculated using the eBay API and past transaction prices of similar watches. The calculated market price is sent to the smartphone, and when the user looks at the watch through the camera, "Market Price: \10,000" is displayed above the watch.

[1184] An example of a prompt sentence might be:

[1185] "Write a program that takes a picture of an old Seiko watch, analyzes the brand, model and condition of the watch using the Google Cloud Vision API, calculates a market price by referencing the eBay API, and uses AR technology to display the market price on top of the watch through the smartphone camera."

[1186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1187] Step 1:

[1188] The user takes a picture of an object using the smartphone camera.

[1189] Input: An image of an object taken by the user.

[1190] Specific operation: The user launches the smartphone camera app and takes a picture of an object they own. For example, if the user has an old watch, they can take a picture of the watch with the camera. When they press the capture button, the image is saved on the smartphone.

[1191] Output: The captured image data.

[1192] Step 2:

[1193] The terminal transmits the captured image to the server.

[1194] Input: The captured image data.

[1195] Specific operation: A smartphone application uploads image data to a server using an HTTP request. For example, the image data is encoded in Base64 format and sent in the body of a POST request.

[1196] Output: Image data sent to the server.

[1197] Step 3:

[1198] The server performs image analysis.

[1199] Input: Image data sent to the server.

[1200] How it works: The server uses image analysis technology such as the Google Cloud Vision API to determine the type and condition of objects in the image. The server sends image data to the API and receives the analysis results in return. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch are determined.

[1201] Output: Analysis results regarding the type and condition of the item.

[1202] Step 4:

[1203] The server calculates the market price of the item in the secondary distribution market.

[1204] Input: Analysis results regarding the type and condition of the item.

[1205] Specific operation: Based on the analysis results, the server refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis results determine that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[1206] Output: The calculated market price.

[1207] Step 5:

[1208] The server transmits the market price to the terminal.

[1209] Input: The calculated market price.

[1210] Specific operation: The server generates JSON data including the market price and returns it to the terminal as an HTTP response. For example, it returns data in the format {"item": "Seiko Watch", "price": "\10,000"}.

[1211] Output: Quote price data sent to the terminal.

[1212] Step 6:

[1213] The device displays market prices using AR technology.

[1214] Input: Quote price data sent to the terminal.

[1215] Specific operation: The smartphone uses AR (Augmented Reality) technology to display the received market price. Specifically, the market price is overlaid on the user's smartphone camera view. For example, when the user looks at their watch through the camera, "Market price: ¥10,000" is displayed on top of the watch. Libraries such as ARKit (iOS) and ARCore (Android) are used for the AR display.

[1216] Output: Quoted prices displayed using AR technology.

[1217] (Application example 1)

[1218] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1219] With conventional systems, it was difficult for users to easily grasp the market price of their items in the secondary market. In addition, in order to check the market price, users had to use multiple websites and applications, which was time-consuming. Furthermore, there was a lack of a way to check the market price in real time when appraising or purchasing items in a physical store.

[1220] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating the market price of the item in the secondary distribution based on the image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing the item based on the displayed market price, a means for photographing the item using the camera of the smart device and determining the type and condition of the item using image analysis technology, a means for acquiring market price data from a database based on the determination result and calculating the market price, a means for displaying the calculated market price on the screen of the smart device using AR technology, and a means for generating a prompt sentence for calculating the market price using a generative AI model. This allows the user to easily check the market price of the item using the smart device and grasp the market price in real time when the item is appraised or purchased in a physical store.

[1221] "Image analysis means" is a technology for analyzing image data acquired through a camera and determining the type and condition of an object.

[1222] The "market price calculation means" is a technology for calculating the market price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[1223] "Means of displaying using AR (Augmented Reality)" refers to technology for overlaying the calculated market price on the screen of a smart device.

[1224] A "means of listing" refers to technology that allows an item to be listed on an online marketplace, etc., based on user operations.

[1225] "Smart device" is a general term for portable electronic devices such as smartphones and tablets that can connect to the Internet and have a camera function.

[1226] "Image analysis technology" is a technology for analyzing image data captured by a camera and extracting specific information.

[1227] "Database" means an information system that stores market data and other related information in an organized manner and enables it to be searched and retrieved as needed.

[1228] A “generative AI model” is a model that is trained to perform a specific task using artificial intelligence, such as in this case to generate prompts for calculating quotes.

[1229] A "prompt sentence" is an input sentence that causes a generative AI model to perform a specific task.

[1230] A system for implementing the present invention uses the following hardware and software.

[1231] Hardware used

[1232] Smart devices (smartphones, tablets, etc.)

[1233] Server (image analysis, database)

[1234] Software used

[1235] Image analysis library (e.g. OpenCV)

[1236] AR library (e.g. ARCore)

[1237] Database (e.g. MySQL)

[1238] Generative AI models (e.g. GPT-4)

[1239] System configuration and operation

[1240] 1. Image capture

[1241] The user uses the camera on their smart device to take a picture of the item they want to appraise, and the image data is temporarily stored on the smart device.

[1242] 2. Image Analysis

[1243] The smart device sends the captured image data to the server, which uses an image analysis library (e.g., OpenCV) to analyze the image data and determine the type and condition of the product.

[1244] 3. Market Calculation

[1245] Based on the results of image analysis, the server retrieves market price data for the relevant item in the secondary distribution market from a database (e.g. MySQL), and calculates the market price based on the retrieved data.

[1246] 4. AR display

[1247] The calculated market price is sent from the server to the smart device, which uses an AR library (e.g., ARCore) to display the market price overlaid on the product seen through the camera.

[1248] 5. Use of generative AI models

[1249] The server uses a generative AI model (e.g. GPT-4) to generate a prompt to calculate the market price. The generated prompt has the following format:

[1250] "What type of item is this, and what is the current going price on the secondary market?"

[1251] Examples

[1252] For example, consider the case where a user wants to purchase a used smartphone at a physical store. The user takes a picture of the product using the smartphone camera. The captured image is sent to the server and analyzed using OpenCV. Based on the analysis results, the market price of the same model in the secondary market is obtained from the database. The calculated market price is displayed on the smartphone screen using ARCore. In this way, the user can easily check the market price when purchasing the product at a physical store.

[1253] This system allows users to easily check the market price of items using their smart devices, and understand the real-time market price when having items appraised or purchased in a physical store.

[1254] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1255] Step 1:

[1256] The user uses the camera of the smart device to take a picture of the item to be appraised. The input is the product image, and the output is the image data temporarily stored in the smart device. In concrete terms, the user starts the camera app, frames the product, and presses the shutter button.

[1257] Step 2:

[1258] The terminal transmits the captured image data to the server. The input is the temporarily stored image data, and the output is the image data transmitted to the server. In concrete terms, the terminal uploads the image data to the server via an Internet connection.

[1259] Step 3:

[1260] The server uses an image analysis library (e.g. OpenCV) to analyze the image data and determine the type and condition of the product. The input is the image data sent to the server, and the output is data about the type and condition of the product. Specifically, the server runs an image analysis algorithm to extract features in the image and identify the product.

[1261] Step 4:

[1262] Based on the results of image analysis, the server retrieves secondary market price data for the relevant product from a database (e.g. MySQL). The input is data about the type and condition of the product, and the output is price data. Specifically, the server executes a database query to search for price information for the relevant product.

[1263] Step 5:

[1264] The server calculates the market price based on the acquired market price data. The input is the market price data, and the output is the calculated market price. Specifically, the server uses statistical methods and algorithms to analyze the market price data and calculate the average price and median price.

[1265] Step 6:

[1266] The server transmits the calculated market price to the smart device. The input is the calculated market price, and the output is the market price transmitted to the smart device. In particular, the server transmits market price data to the smart device via an internet connection.

[1267] Step 7:

[1268] The device uses an AR library (e.g. ARCore) to overlay the market price on the product seen through the camera. The input is the market price sent from the server, and the output is the market price displayed on the smart device screen. In concrete terms, the device overlays the market price on the camera image.

[1269] Step 8:

[1270] The server uses a generative AI model (e.g. GPT-4) to generate a prompt for calculating the market price. The input is data on the type and condition of the product, and the output is the generated prompt. In concrete terms, the server requests the generative AI model to generate a prompt, and obtains the following prompt:

[1271] "What type of item is this, and what is the current going price on the secondary market?"

[1272] In this way, users can easily check the market price of items using their smart devices and understand the market price in real time when having an item appraised or purchased in a physical store.

[1273] Example 2

[1274] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1275] With conventional systems, it was difficult to accurately calculate the market price of goods in the secondary distribution market and provide an environment where users could easily list items for sale. In addition, there was a lack of a way to effectively utilize past transaction data when calculating market prices, so it was not possible to provide users with reliable market price information. Furthermore, the listing process was complicated, placing a heavy burden on users.

[1276] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating a market price in the secondary distribution of an item based on image data obtained by the image analysis means, a means for calculating a market price using a generative AI model, a means for displaying the calculated market price, a means for listing an item based on the displayed market price, a means for inputting listing information based on a user's operation, a means for transmitting the input listing information to the server, and a means for storing the listing information received by the server in a database. This enables a user to easily list an item based on highly reliable market price information.

[1277] An "image analysis means" is a device or software that analyzes image data acquired through a camera and recognizes the characteristics and condition of an object.

[1278] The "market price calculation means" refers to a device or software for calculating the price of an item in the secondary market based on the data obtained by the image analysis means and past transaction data.

[1279] A "generative AI model" is a model that uses artificial intelligence techniques to analyze data and perform specific tasks, in this case to calculate quotes.

[1280] A "means for displaying quotes" is a device or software for visually presenting the calculated quotes to a user.

[1281] A "means for listing" is a device or software that supports a series of operations for a user to list an item on the secondary market.

[1282] "Means for inputting listing information" refers to a device or software that allows a user to input information such as the price and comments of the item being listed.

[1283] The "means for transmitting the selling information to the server" refers to a device or software for transmitting the selling information input by the user to the server.

[1284] The "means for storing in a database" refers to a device or software that stores the listing information received by the server in a database and makes it available for other users to view.

[1285] This invention is a system that allows a user to list items for sale using a smartphone, and includes an image analysis means, a means for calculating a market price, a generative AI model, a means for displaying a market price, a means for listing an item, a means for inputting listing information, a means for transmitting the listing information to a server, and a means for saving the listing information in a database.

[1286] The user starts the app on their smartphone and uses the camera to capture an image of the item they wish to sell. The image analysis means analyzes the captured image data and recognizes the characteristics and condition of the item. The analyzed data is then sent to the server.

[1287] The server retrieves historical trading data from a database (e.g., MySQL) and calculates quotes using a generative AI model (e.g., OpenAI's GPT-4). For example, it sends a prompt like the following to the generative AI model:

[1288] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[1289] Transaction 1: 400 yen

[1290] Transaction 2: 500 yen

[1291] Transaction 3: 600 yen

[1292] The generative AI model calculates the market price based on this data, and the server sends the calculated market price to the user's device.

[1293] The terminal displays the received market price on the screen. For example, it may say, "The market price for this book is 500 yen." The user selects the item to sell based on the displayed market price, and inputs the selling price and listing comment on the listing screen. For example, the user sets the selling price to 600 yen and inputs the comment, "The item is in good condition."

[1294] When the user presses the sell button, the terminal sends the entered selling information to the server, which then stores the received selling information in a database and makes it available for other users to view.

[1295] The system allows users to easily list items based on reliable market information, and by using generative AI models, it effectively leverages past transaction data to provide accurate market prices.

[1296] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1297] Step 1:

[1298] The user launches the app.

[1299] A user launches an app by tapping the app icon on the home screen of their smartphone. The input is the user's operation, and the output is the launch of the app.

[1300] Step 2:

[1301] A user uses a camera to capture an image of an object.

[1302] A user uses the camera function in the app to take a picture of the item they want to put up for sale. The input is the image data acquired through the camera, and the output is the image data sent to the image analysis means.

[1303] Step 3:

[1304] The server analyzes the image data.

[1305] The server uses image analysis means to analyze the acquired image data and recognize the features and state of the object. The input is the image data sent by the user, and the output is the analyzed feature data of the object.

[1306] Step 4:

[1307] The server calculates the market price.

[1308] The server retrieves historical trading data from the database and uses the generative AI model to calculate quotes. For example, it sends the following prompt to the generative AI model:

[1309] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[1310] Transaction 1: 400 yen

[1311] Transaction 2: 500 yen

[1312] Transaction 3: 600 yen

[1313] The input is historical trading data and a prompt, and the output is the calculated quote price.

[1314] Step 5:

[1315] The server transmits the market price to the terminal.

[1316] The server sends the calculated quotes to the user's terminal. The input is the calculated quote and the output is the quote sent to the terminal.

[1317] Step 6:

[1318] The terminal will display the market price.

[1319] The terminal displays the received market price on the screen. For example, it displays "The market price of this book is 500 yen." The input is the market price sent from the server, and the output is the market price displayed on the screen.

[1320] Step 7:

[1321] The user selects the item to sell.

[1322] The user taps the item they want to sell from the list of market prices displayed on the screen. The input is the user's operation, and the output is a detailed screen of the selected item.

[1323] Step 8:

[1324] The device displays the listing screen.

[1325] The terminal opens a listing screen that displays detailed information about the selected item. The input is the information about the item selected by the user, and the output is the listing screen.

[1326] Step 9:

[1327] The user enters listing information.

[1328] The user inputs the listing price and listing comment on the listing screen. For example, the listing price is set to 600 yen and the comment "Condition is good." The input is the listing information entered by the user, and the output is the entered listing information.

[1329] Step 10:

[1330] The user presses the sell button.

[1331] The user presses the "list" button to confirm the information they have entered. The input is the user's operation, and the output is the confirmed listing information.

[1332] Step 11:

[1333] The terminal transmits the listing information to the server.

[1334] The terminal transmits the inputted selling information to the server. The input is the confirmed selling information, and the output is the selling information transmitted to the server.

[1335] Step 12:

[1336] The server stores the listing information in a database.

[1337] The server stores the received listings in a database and makes them available for other users to view. The input is the listing sent to the server and the output is the listing stored in the database.

[1338] (Application example 2)

[1339] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1340] In the conventional secondary distribution of goods, it was difficult for users to grasp the appropriate market price, and the listing process was cumbersome. In addition, it was time-consuming for users to input the listing price and comments, and the listing information was not saved efficiently. This reduced user convenience and hindered the revitalization of the secondary distribution market.

[1341] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1342] In this invention, the server includes an image analysis means, a means for calculating a market price for an item in secondary distribution based on image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing an item based on the displayed market price, and a means for a user to input a listing price and a comment and store listing information. This allows a user to easily grasp an appropriate market price and efficiently list items. Furthermore, since the listing information is efficiently stored, user convenience is improved and the secondary distribution market is expected to be revitalized.

[1343] An "image analysis means" is a device or software that analyzes image data acquired through a camera and extracts features and attributes of objects.

[1344] The "market price calculation means" refers to a device or software that calculates the fair price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[1345] "Means of displaying using AR (Augmented Reality)" refers to a technology that displays the calculated market price by overlaying it on a real-world image, and is a device or software that allows users to visually confirm the market price.

[1346] "Means for listing" refers to a device or software that provides an interface for users to list items based on market prices and allows for the input and storage of listing information.

[1347] The "means for inputting the listing price and comments and storing the listing information" refers to a device or software that allows the user to input the listing price and comments and stores the information in a database or the like.

[1348] A system for carrying out this invention has the following configuration. First, a user takes an image of an object using a smartphone. The smartphone is equipped with a camera, and image data acquired through this camera is sent to image analysis means. The image analysis means analyzes the image data and extracts features and attributes of the object.

[1349] Next, based on the data obtained by the image analysis means, the server calculates the fair price of the item in the secondary distribution market. This is the means of calculating the market price. The calculated market price is displayed on the smartphone display using AR (Augmented Reality) technology. This allows the user to visually check the market price by overlaying it on an image of the real world.

[1350] After the user checks the market price, an interface for entering the listing price and comments will be displayed on the smartphone screen. The user enters the listing price and comments through this interface and saves the listing information by pressing the listing button. This is the means for listing and saving the listing information.

[1351] The hardware used is a smartphone, and the software uses Flask (a Python web framework). Data processing and calculation involve obtaining market prices and storing listing information. Specifically, computer vision technology is used for image analysis, and machine learning models are used to calculate market prices.

[1352] As a concrete example, the prompt text when a user lists "item1" is as follows:

[1353] Example prompt:

[1354] A user is trying to list "item1". Get the market price, set the listing price to 1500 yen, and set the comment to "Brand new, unused."

[1355] In this way, users can easily check market prices and efficiently carry out the listing process. Furthermore, because listing information is efficiently stored, user convenience is improved and the secondary distribution market is expected to become more active.

[1356] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1357] Step 1:

[1358] A user takes an image of an object using the smartphone camera.

[1359] Input: Image of thing

[1360] Output: Image data

[1361] Specific operation: A user starts the camera app on their smartphone and takes a picture of an object. The captured image data is saved in the smartphone.

[1362] Step 2:

[1363] The terminal uses image analysis means to analyze the captured image data.

[1364] Input: Image data

[1365] Output: Feature and attribute data of objects

[1366] How it works: Image analysis software in the smartphone processes image data and extracts the object's features (e.g., shape, color, size) and attributes (e.g., brand, condition).

[1367] Step 3:

[1368] The server calculates the market price of the item in the secondary distribution market based on the data obtained by the image analysis means.

[1369] Input: Features and attribute data of objects

[1370] Output: Market price data

[1371] Specific operation: The server uses a machine learning model to input the item's characteristics and attributes, and calculates the market price based on past transaction data and market data.

[1372] Step 4:

[1373] The device displays the calculated market price using AR (augmented reality) technology.

[1374] Input: Market price data

[1375] Output: AR displayed market price

[1376] How it works: The smartphone's AR app receives market price data and displays the market price by overlaying it on real-world images via the camera.

[1377] Step 5:

[1378] The user enters the listing price and comments on the smartphone screen.

[1379] Input: Market price data, user-entered listing prices and comments

[1380] Output: Listing information

[1381] Specific operation: The user enters the listing price and comments into the listing interface on the smartphone and presses the listing button. The entered information is compiled into the listing information.

[1382] Step 6:

[1383] The server stores the listing information.

[1384] Input: Listing information

[1385] Output: Saved listings

[1386] Specific operation: The server saves the listing information to the database and notifies the user that the listing is complete.

[1387] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1388] "Example 1"

[1389] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine recognizes emotions from the user's facial expressions and tone of voice. Specifically, the user takes a picture of his or her face through the smartphone camera, and recognizes emotions such as joy, sadness, surprise, and anger from the facial expressions. In addition, the user records his or her voice through the smartphone microphone, and recognizes emotions from the tone of the voice.

[1390] "Example 2"

[1391] Furthermore, the emotion engine recommends listing items based on the user's emotions. Specifically, if a user expresses an emotion of happiness, it recommends listing items that make the user happy. Conversely, if a user expresses an emotion of sadness, it recommends listing items that make the user sad. In this way, the emotion engine understands the user's emotions and helps the user list items based on them.

[1392] The process flow of each embodiment will be described below.

[1393] "Example 1"

[1394] Step 1: The user takes a picture of their face using their smartphone camera.

[1395] Step 2: The emotion engine recognizes emotions such as happiness, sadness, surprise, and anger from the captured facial expressions.

[1396] Step 3: The user records their voice through the smartphone's microphone.

[1397] Step 4: The emotion engine identifies emotions from the tone of the recorded voice.

[1398] "Example 2"

[1399] Step 1: The emotion engine recognizes the user's emotion.

[1400] Step 2: If the user expresses the emotion of joy, the emotion engine will recommend listing something that brings the user joy.

[1401] Step 3: If the user indicates the emotion of sadness, the emotion engine will recommend that the user list something that makes them sad.

[1402] Example 1

[1403] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1404] In the conventional system, it was difficult for users to easily grasp the secondary market price of the items they owned, and the information provided did not take into account the users' feelings. As a result, it was difficult for users to make decisions when listing items, and there was an issue that the user experience did not improve.

[1405] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1406] In this invention, the server includes an image analysis means, a means for calculating the secondary market price of the item, a means for displaying the market price in augmented reality, a means for recognizing the user's emotions from their facial expressions and tone of voice, and a means for providing information to the user based on the recognized emotions. This allows the user to easily grasp the secondary market price of the item they own, and to receive appropriate information according to their emotions.

[1407] The "image analysis means" is a means for analyzing image data acquired through a camera and determining the type and condition of an object.

[1408] The "means for calculating the market price in the secondary distribution of goods" refers to a means for calculating the trading price in the secondary distribution market of goods based on the data obtained by the image analysis means.

[1409] "Means for displaying using augmented reality" refers to means for visually displaying the calculated market price to the user using augmented reality technology.

[1410] "Means for recognizing emotions from a user's facial expressions and tone of voice" refers to means for analyzing a user's facial expressions and tone of voice to identify emotions.

[1411] The "means for providing information to a user based on a recognized emotion" refers to a means for providing appropriate information to a user in accordance with the emotion of the user identified by the emotion recognition means.

[1412] A "means for listing items" is a means for a user to list items they own on the secondary market based on the displayed market price.

[1413] The present invention is a system that allows a user to easily grasp the secondary market price of an item owned by the user and provides information according to the user's emotions. A specific embodiment of this system will be described below.

[1414] System Configuration

[1415] This system consists of a user's terminal (smartphone), a server, an image analysis means, a market price calculation means, an augmented reality display means, an emotion recognition means, an information provision means, and an listing means.

[1416] Hardware and Software

[1417] Device (smartphone): Camera, microphone, AR app (e.g. ARKit, ARCore)

[1418] Server: Image analysis software (e.g. Google Cloud Vision API), market price calculation software (e.g. eBay API, Amazon API), emotion recognition software (e.g. Microsoft Azure Emotion API)

[1419] Processing flow

[1420] 1. The user takes a photo of an object

[1421] Users use the smartphone camera to take a picture of an object they own. For example, if a user wants to take a picture of an old book, they start the smartphone camera app and take a picture of the book cover.

[1422] 2. The device sends the image to the server

[1423] The device sends the captured image to the server. At this time, the device uses an Internet connection to upload the image data to the server.

[1424] 3. The server analyzes the image

[1425] The server analyzes the received images. Specifically, the server uses the Google Cloud Vision API to determine the type and condition of the objects in the image. For example, the server can identify the title, author, and condition of a book (new, used, damaged, etc.).

[1426] 4. The server calculates the secondary market price of the item

[1427] Based on the results of the search, the server calculates the market price of the item in the secondary market. To do this, the server refers to the eBay API or Amazon API to check the past transaction prices of similar items. For example, the server calculates the market price by averaging the transaction prices of the same book over the past year.

[1428] 5. The server sends the quotes to the terminal

[1429] The server then sends the calculated market price to the device, which receives the data and displays the market price using an AR app.

[1430] 6. The device displays market prices in AR

[1431] The device uses ARKit and ARCore to display a price quote on an object when the user looks at it through the smartphone camera. For example, when the user looks at a book through the camera, "Price quote: $1,500" is displayed on the book.

[1432] 7. Recognize user emotions

[1433] The user uses the smartphone camera to take a picture of their face and the smartphone microphone to record their voice, and the device transmits this data to a server.

[1434] 8. The server analyzes emotions

[1435] The server uses the Microsoft Azure Emotion API to identify emotions from the user's facial expressions. The server also recognizes emotions from the tone of the recorded voice. For example, the server recognizes a user's smile as "happiness" and a high-pitched voice as "excitement."

[1436] 9. The server sends the emotion recognition results to the device.

[1437] The server sends the emotion recognition results to the device, which receives the data and displays it to the user.

[1438] 10. The device displays the emotion recognition results.

[1439] The device will display the emotion recognition results to the user. For example, if the user is smiling, it will display "You are happy."

[1440] Examples

[1441] Example 1: Finding the market value of old books

[1442] 1. The user takes a photo of an old book with their smartphone camera.

[1443] 2. The device sends the image to the server.

[1444] 3. The server uses the Google Cloud Vision API to identify the book's title and status.

[1445] 4. The server calculates the market price by referencing the eBay API or Amazon API.

[1446] 5. The server sends the market price to the terminal.

[1447] 6. The device uses ARKit to display price quotes on top of the book.

[1448] Example 2: Emotion recognition

[1449] 1. The user takes a photo of their face with their smartphone camera and records their voice with the microphone.

[1450] 2. The device sends this data to the server.

[1451] 3. The server uses the Microsoft Azure Emotion API to identify emotions from facial expressions.

[1452] 4. The server recognizes emotions from the tone of the recorded voice.

[1453] 5. The server sends the emotion recognition results to the device.

[1454] 6. The device displays the emotion recognition results to the user.

[1455] Examples of prompt statements

[1456] "I want to find out the market price of an old book. I took a picture of the book with my smartphone camera. Please tell me the market price."

[1457] "Please recognize my emotions. I photographed my face with my phone camera and recorded my voice with a microphone."

[1458] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1459] Step 1:

[1460] The user takes a photo of an object

[1461] A user uses the smartphone camera to take a picture of an object they own. For example, if a user wants to take a picture of an old book, they start the smartphone camera app and take a picture of the book's cover. The input is the image of the object, and the output is the captured image data.

[1462] Step 2:

[1463] The device sends the image to the server.

[1464] The terminal sends the captured image to the server. At this time, the terminal uses an Internet connection to upload image data to the server. The input is the captured image data, and the output is the image data sent to the server.

[1465] Step 3:

[1466] The server analyzes the image

[1467] The server analyzes the images it receives. Specifically, the server uses image analysis software (e.g., Google Cloud Vision API) to determine the type and condition of the object in the image. For example, the server might identify the book's title, author, and condition (new, used, damaged, etc.). The input is the image data sent to the server, and the output is data about the type and condition of the object.

[1468] Step 4:

[1469] The server calculates the secondary market price of the item

[1470] Based on the results of the determination, the server calculates the market price of the item in the secondary market. To do this, the server refers to a database (e.g., eBay API or Amazon API) and investigates past transaction prices of similar items. For example, the server calculates the market price by averaging the transaction prices of the same book over the past year. The input is data about the type and condition of the item, and the output is the calculated market price.

[1471] Step 5:

[1472] The server sends the market price to the terminal.

[1473] The server sends the calculated market price to the terminal. The terminal receives this data and displays the market price using the AR app. The input is the calculated market price, and the output is the market price data sent to the terminal.

[1474] Step 6:

[1475] The device displays market prices in AR

[1476] The device uses ARKit and ARCore to display a current price on top of an object when the user looks at it through the smartphone camera. For example, when the user looks at a book through the camera, "Current Price: $1,500" is displayed on top of the book. The input is the current price data sent to the device, and the output is the current price displayed in AR.

[1477] Step 7:

[1478] Recognize user emotions

[1479] The user takes a picture of their face using the smartphone camera and records their voice using the smartphone microphone. The device sends these data to the server. The input is the user's face image data and voice recording data, and the output is emotion recognition data sent to the server.

[1480] Step 8:

[1481] The server analyzes emotions

[1482] The server uses emotion recognition software (e.g., Microsoft Azure Emotion API) to identify emotions from the user's facial expressions. The server also recognizes emotions from the tone of a recorded voice. For example, the server may recognize a user's smiling face as "happiness" and a high-pitched voice as "excitement." The input is the emotion recognition data sent to the server, and the output is the recognized emotion data.

[1483] Step 9:

[1484] The server sends the emotion recognition results to the device.

[1485] The server sends the emotion recognition results to the terminal. The terminal receives this data and displays it to the user. The input is the recognized emotion data, and the output is the emotion recognition result data sent to the terminal.

[1486] Step 10:

[1487] The device displays the emotion recognition results.

[1488] The terminal displays the emotion recognition result to the user. For example, if the user is smiling, "You are happy" is displayed. The input is the emotion recognition result data sent to the terminal, and the output is the emotion recognition result displayed to the user.

[1489] (Application example 1)

[1490] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1491] In the conventional secondary market, users had to spend a lot of time and effort to understand the market price of the items they own. In addition, there was also the issue that the user experience did not improve because information was provided uniformly without considering the user's feelings.

[1492] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1493] In this invention, the server includes an image analysis means, a means for calculating a market price in secondary distribution of an item based on image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing an item based on the displayed market price, an emotion recognition means for recognizing an emotion from a user's facial expression and tone of voice, and a means for providing feedback based on the recognized emotion. This allows a user to quickly and easily grasp the market price of an item that he or she owns, and to receive appropriate feedback according to the emotion.

[1494] The "image analysis means" is a means for analyzing image data acquired through a camera and determining the type and condition of an object.

[1495] The "market price calculation means" is a means for calculating the market price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[1496] "Means for displaying using AR (Augmented Reality)" refers to a means for displaying the calculated market price by overlaying it on a real image.

[1497] A "means for listing items" is a means for a user to list items on the secondary market based on the displayed market price.

[1498] An "emotion recognition means" is a means for analyzing a user's facial expressions and tone of voice to recognize the user's emotions.

[1499] The "means for providing feedback" is a means for providing appropriate feedback to a user based on the recognized user sentiment.

[1500] A system for implementing this invention has the following configuration. First, a user takes a picture of an item they own using the smartphone camera. The captured image is analyzed by image analysis means to determine the type and condition of the item. Next, the market price of the item in the secondary distribution market is calculated based on the results of the determination. The calculated market price is displayed to the user using AR (Augmented Reality) technology. Specifically, when the user looks at an item through the smartphone camera, the market price is displayed above the item.

[1501] Furthermore, an emotion recognition means is incorporated, and emotions are recognized from the user's facial expressions and tone of voice. Specifically, the user takes a picture of his / her face using the smartphone camera, and emotions such as joy, sadness, surprise, and anger are recognized from the facial expressions. In addition, the user records his / her voice using the smartphone microphone, and emotions are recognized from the tone of the voice. Based on the recognized emotions, the feedback providing means provides appropriate feedback to the user.

[1502] Hardware and Software Used:

[1503] Hardware: Smartphone (camera, microphone)

[1504] Software: OpenCV (image processing library), Keras (deep learning library for image analysis and emotion recognition), pyttsx3 (voice synthesis library), dlib (face detection library)

[1505] Data processing and calculation:

[1506] 1. Image analysis: Images taken with a smartphone camera are read using OpenCV, and the type and condition of the item are determined using a Keras model, and the market price in the secondary market is calculated.

[1507] 2. Emotion recognition: Face detection is performed using dlib on images of the user's face taken with a smartphone camera, and emotions are recognized using a Keras model.

[1508] 3. AR display: Using OpenCV, the calculated market price is displayed on the image.

[1509] 4. Audio feedback: Use pyttsx3 to provide audio feedback according to the recognized emotion.

[1510] Examples:

[1511] When users take a picture of their old phone with their smartphone camera, the phone's market value on the secondary market is displayed on the screen, and if the user smiles, voice feedback is provided saying, "This item has good resale value."

[1512] Example prompt for a generative AI model:

[1513] Analyze the images taken by the user using the smartphone camera and calculate the market price of the item in the secondary market. Also, recognize the emotions of the user from their facial expressions and tone of voice and provide feedback according to their emotions.

[1514] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1515] Step 1:

[1516] A user takes a picture of an object they own using the camera of their smartphone. The input is the image data acquired through the camera, and the output is that image data. In concrete terms, the user starts the camera app on their smartphone, puts the object in the frame, and presses the capture button.

[1517] Step 2:

[1518] The terminal transmits the captured image data to the image analysis means. The input is the image data acquired in step 1, and the output is the data transmitted to the image analysis means. In concrete terms, the smartphone application transmits the image data to the server.

[1519] Step 3:

[1520] The server uses image analysis means to analyze the image data and determine the type and status of the object. The input is the image data sent in step 2, and the output is data related to the type and status of the object. In concrete terms, an image analysis model (using Keras) on the server processes the image data and determines the type and status of the object.

[1521] Step 4:

[1522] The server calculates the market price of the item in the secondary market based on the judgment result. The input is the data on the type and condition of the item obtained in step 3, and the output is the market price. In concrete terms, the server refers to the database and calculates the market price based on the market prices of similar items.

[1523] Step 5:

[1524] The server sends the calculated market price to the terminal, which then displays the market price using AR technology. The input is the market price calculated in step 4, and the output is the market price displayed on the smartphone screen. In concrete terms, the smartphone application overlays the market price on the camera image.

[1525] Step 6:

[1526] A user takes a picture of his / her face through the smartphone camera, and the terminal transmits the image data to the emotion recognition means. The input is the face image data acquired through the camera, and the output is the data transmitted to the emotion recognition means. In concrete terms, the user points the smartphone camera at himself / herself and takes a picture of his / her face.

[1527] Step 7:

[1528] The server uses emotion recognition means to recognize emotions from the user's facial expressions and tone of voice. The input is the facial image data and voice data sent in step 6, and the output is the recognized emotion data. In concrete terms, the emotion recognition model (using Keras) on the server analyzes the facial image and voice data to determine the emotion.

[1529] Step 8:

[1530] The server generates feedback based on the recognized emotion and sends it to the terminal. The input is the emotion data obtained in step 7, and the output is a feedback message. In concrete terms, the server generates a feedback message according to the emotion data and sends it to the terminal.

[1531] Step 9:

[1532] The terminal provides the feedback message to the user using a speech synthesis engine. The input is the feedback message sent in step 8, and the output is the voice feedback provided to the user. In concrete terms, the smartphone application uses pyttsx3 to play back the feedback message aloud.

[1533] Example 2

[1534] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1535] In conventional secondary distribution systems, it was difficult for users to grasp the appropriate market price, and listing recommendations were not made based on the user's emotions. This made it difficult for users to list items at the optimal time, which resulted in the revitalization of the secondary distribution market being hindered. Furthermore, because listing recommendations were not made based on the user's emotions, there was a possibility that user satisfaction would decrease.

[1536] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1537] In this invention, the server includes an image analysis means, a market price calculation means, a market price display means, a listing means, an emotion analysis means, and a listing recommendation means. This allows the user to understand the appropriate market price and receive listing recommendations based on emotions. This allows the user to list items at the optimal time, promoting the revitalization of the secondary distribution market. Furthermore, listing recommendations that take the user's emotions into consideration improve user satisfaction.

[1538] The "image analysis means" refers to a means for analyzing image data acquired using a camera or other image acquisition device, and extracting necessary information.

[1539] The "market price calculation means" is a means for calculating the market price in the secondary distribution market of an item based on the data obtained by the image analysis means.

[1540] The "market price display means" is a means for visually displaying the calculated market price to the user.

[1541] A "listing means" is a means by which a user lists an item on the secondary distribution market.

[1542] The "emotion analysis means" is a means for analyzing the user's emotional data and determining the user's emotional state.

[1543] The "item recommendation means" is a means for recommending appropriate items to a user for sale based on the user's emotion data analyzed by the emotion analysis means.

[1544] This invention is a system that allows a user to grasp an appropriate market price when putting an item up for sale on a secondary market, and further allows the user to receive a recommendation for listing based on the user's emotions. This system includes an image analysis means, a market price calculation means, a market price display means, a listing means, an emotion analysis means, and a listing recommendation means.

[1545] Hardware and Software Used

[1546] Hardware: Smartphones, servers

[1547] Software: Mobile applications, emotion engines, generative AI models

[1548] Program Processing

[1549] 1. Obtaining and displaying market prices

[1550] The server retrieves the market price of the item from the database by executing a SQL query like "SELECT FROM market_prices WHERE item_id = 'item123'".

[1551] The server sends the obtained market price to the mobile application on the smartphone. Specifically, the server uses the REST API to send a request such as "GET / api / market_prices / item123".

[1552] The terminal (smartphone) displays the received market price on the screen. Specifically, the terminal executes code such as "item_price_text_view.setText('1000 yen')".

[1553] 2. Listing items

[1554] The user taps on the smartphone screen where the market price is displayed. Specifically, the user taps on the part of the smartphone case where the market price is displayed.

[1555] The device displays the listing screen for the tapped item. Specifically, the device executes code such as "startActivity(new Intent(this, ListingActivity.class))".

[1556] The user inputs the listing price and listing comment on the listing screen and presses the listing button. Specifically, the user inputs "Listing price: 1000 yen" and "Listing comment: Brand new, unused" and presses the "List" button.

[1557] The terminal sends the entered information to the server. Specifically, the terminal sends a request such as "POST / api / listings" and includes the listing information in the body.

[1558] The server stores the received information in a database and lists the item. Specifically, the server executes a SQL query like "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'item123')".

[1559] 3. Listing recommendations based on emotion engine

[1560] The emotion engine analyzes the user's emotion data. Specifically, the emotion engine executes a function such as "analyzeEmotion(user_data)".

[1561] When a user expresses the emotion of joy, the emotion engine recommends listing items that the user feels joy in. Specifically, the emotion engine executes a function such as "recommendItem('joy')" to determine the recommended items.

[1562] If a user expresses an emotion of sadness, the emotion engine recommends listing items that make the user feel sad. Specifically, the emotion engine executes a function such as "recommendItem('sadness')" to determine the recommended items.

[1563] The device notifies the user of the recommendations from the emotion engine. Specifically, the device executes a code such as "showNotification('There are recommended items for sale')".

[1564] Examples

[1565] Obtaining and displaying market prices

[1566] The server executes the SQL query "SELECT FROM market_prices WHERE item_id = 'smartphone_case'" and obtains the market price of "1,000 yen."

[1567] The server sends a "GET / api / market_prices / smartphone_case" request to the smartphone.

[1568] The terminal executes the code "item_price_text_view.setText('1000 yen')" and displays "1000 yen" on the screen.

[1569] Selling items

[1570] When the user taps on the market price for "Smartphone Case", the device executes the "startActivity(new Intent(this, ListingActivity.class))" code and displays the listing screen.

[1571] The user enters "Listing price: 1,000 yen" and "Listing comment: Brand new, unused" and presses the "List" button.

[1572] The terminal sends a "POST / api / listings" request to the server and includes the listing information in the body.

[1573] The server executes the SQL query "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'smartphone_case')" to complete the listing.

[1574] Listing recommendations based on emotion engine

[1575] The emotion engine executes the function “analyzeEmotion(user_data)” and determines that the user is expressing the emotion of happiness.

[1576] The emotion engine executes the function "recommendItem('joy')" and determines that "smartphone case" is the recommended item.

[1577] The terminal executes the code "showNotification('There are recommended items for sale')" to notify the user.

[1578] Examples of prompt statements

[1579] "When a user expresses the emotion of joy, what items do you recommend them to sell?"

[1580] "When a user expresses sadness, what items do you recommend they list?"

[1581] In this way, the system helps users list items based on their sentiment and market prices.

[1582] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1583] Step 1:

[1584] Get market prices

[1585] The server retrieves the market price of the item from the database.

[1586] Input: Thing ID (e.g. 'item123')

[1587] Data processing: The server executes the SQL query "SELECT FROM market_prices WHERE item_id = 'item123'".

[1588] Output: Market price data (e.g. '1000 yen')

[1589] Step 2:

[1590] Sending quotes

[1591] The server transmits the obtained market price to a mobile application on the smartphone.

[1592] Input: Market price data (e.g. '1000 yen')

[1593] Data processing: The server uses the REST API to send a request for “GET / api / market_prices / item123”.

[1594] Output: Quote price data is sent to your smartphone.

[1595] Step 3:

[1596] Display of market price

[1597] The terminal (smartphone) displays the received market price on the screen.

[1598] Input: Market price data (e.g. '1000 yen')

[1599] Data processing: The terminal executes the code "item_price_text_view.setText('1000 yen')".

[1600] Output: The market price is displayed on the smartphone screen.

[1601] Step 4:

[1602] Selection of items

[1603] The user taps on the item whose market price is displayed on the smartphone screen.

[1604] Input: User taps

[1605] Data processing: The device obtains the ID of the tapped object.

[1606] Output: The ID of the item (e.g. 'item123')

[1607] Step 5:

[1608] Display of listing screen

[1609] The device displays the listing screen for the item that was tapped.

[1610] Input: Thing ID (e.g. 'item123')

[1611] Data processing: The device executes the code "startActivity(new Intent(this, ListingActivity.class))".

[1612] Output: The listing screen is displayed.

[1613] Step 6:

[1614] Entering listing information

[1615] The user inputs the listing price and listing comment on the listing screen and presses the listing button.

[1616] Input: Listing price (e.g. '1000 yen'), listing comment (e.g. 'Brand new, never used')

[1617] Data processing: The terminal retrieves the entered information.

[1618] Output: Listing information (e.g. '1000 yen', 'Brand new, never used')

[1619] Step 7:

[1620] Submit your listing

[1621] The terminal transmits the input information to the server.

[1622] Input: Listing information (e.g. '1000 yen', 'Brand new, never used')

[1623] Data processing: The terminal sends a “POST / api / listings” request and includes the listing information in the body.

[1624] Output: The listing is sent to the server.

[1625] Step 8:

[1626] Completing the listing

[1627] The server stores the received information in a database and lists the item.

[1628] Input: Listing information (e.g. '1000 yen', 'Brand new, never used')

[1629] Data processing: The server executes the SQL query "INSERT INTO listings (price, comment, item_id) VALUES ('1000', 'Brand new', 'item123')".

[1630] Output: The listing is completed and saved to the database.

[1631] Step 9:

[1632] Emotion data analysis

[1633] The emotion engine analyzes the user's emotion data.

[1634] Input: User emotion data (e.g. 'joy')

[1635] Data processing: The emotion engine executes the function “analyzeEmotion(user_data)”.

[1636] Output: Sentiment analysis result (e.g. 'joy')

[1637] Step 10:

[1638] Decision on recommended products

[1639] When a user expresses the emotion of joy, the emotion engine recommends listing items that bring joy to the user.

[1640] Input: Sentiment analysis result (e.g. 'joy')

[1641] Data processing: The emotion engine executes the function “recommendItem('joy')” to determine the recommended items.

[1642] Output: Recommended item (e.g. 'phone case')

[1643] Step 11:

[1644] Recommendation Notification

[1645] The terminal notifies the user of the recommendations from the emotion engine.

[1646] Input: Recommended item (e.g. 'phone case')

[1647] Data processing: The terminal executes the code "showNotification('There are recommended items for sale')".

[1648] Output: A notification is displayed to the user.

[1649] (Application example 2)

[1650] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1651] In conventional secondary distribution systems, it was difficult for users to grasp the market price when listing items, and listing recommendations were not based on users' emotions. This made it difficult for users to list items at the appropriate price, and did not increase their motivation to list items.

[1652] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1653] In this invention, the server includes image analysis means, means for calculating a market price in secondary distribution of an item based on image data obtained by the image analysis means, means for displaying the calculated market price in augmented reality, means for listing an item based on the displayed market price, emotion recognition means for recognizing an emotion of a user, and means for recommending listing based on the recognized emotion. This enables a user to list an item at an appropriate price, and furthermore, since listing recommendations are made based on the emotion of the user, it is possible to increase the motivation to list an item.

[1654] An "image analysis means" is a device or software that analyzes image data acquired through a camera and recognizes the characteristics and condition of an object.

[1655] The "market price calculation means" refers to a device or software for calculating the price of an item in the secondary market based on the data obtained by the image analysis means.

[1656] "Means for displaying in augmented reality" refers to a device or software for displaying the calculated market price overlaid on a real-world image.

[1657] A "means for listing" is a device or software that allows a user to list an item based on the displayed market price.

[1658] An "emotion recognition means" is a device or software for analyzing and recognizing emotions from a user's facial expressions, voice, etc.

[1659] The "means for recommending items" is a device or software for suggesting appropriate items for sale based on the recognized user's sentiment.

[1660] The following system configuration will be described as an embodiment of the present invention.

[1661] First, the server is equipped with an image analysis means and analyzes image data acquired through the camera. This image analysis means includes a means for recognizing the characteristics and state of the object, and calculating the market price of the object in the secondary distribution market based on the data. The market price calculation means calculates the market price based on the acquired data, and transmits the result to the means for displaying in augmented reality.

[1662] Next, the terminal is provided with a means for displaying in augmented reality, and displays the calculated market price by overlaying it on a real image. Based on this display, the user can put the item up for sale. The means for listing provides an interface for listing the item based on the user's operation.

[1663] Furthermore, the server is equipped with an emotion recognition means for recognizing the emotion of the user. This emotion recognition means analyzes and recognizes the emotion of the user from facial expressions, voice, etc. Based on the recognized emotion, the listing recommendation means suggests listing of appropriate items.

[1664] As a specific example, when a user takes a picture of an item using a smartphone camera, the image data is sent to a server. The server analyzes the item using image analysis means and calculates a market price. The market price is displayed in augmented reality on the smartphone screen. When the user puts up an item for sale based on the price, emotion recognition means analyzes the user's emotions, and if the user is expressing an emotion of joy, for example, a message encouraging the listing is displayed.

[1665] The following hardware and software are used to realize this system:

[1666] Hardware: Smartphone camera

[1667] Software: OpenCV (image analysis library), EmotionRecognizer (emotion recognition library), market_data (market price data acquisition library), ecommerce_platform (mail order site API)

[1668] An example of a prompt sentence might be:

[1669] Develop an application that allows users to sell items using their smartphone camera. The application has the following features:

[1670] 1. Capture footage from the camera and recognize the user's emotions.

[1671] 2. Generate listing recommendations based on the perceived sentiment.

[1672] 3. Use the Market Data Acquisition library to obtain quotes for the selected instruments.

[1673] 4. The user is shown the listing screen and asked to enter the listing price and comments.

[1674] 5. Use the online shopping site's API to list your products.

[1675] In this way, users are able to list items at appropriate prices, and recommendations for listings are made based on the user's emotions, which increases the motivation to list items.

[1676] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1677] Step 1:

[1678] A user takes a picture of a product using the camera of a smartphone. The input is the video data from the camera, and the output is image data. This image data is sent to the server.

[1679] Step 2:

[1680] The server uses image analysis means to analyze the received image data. The input is image data, and the output is data indicating the characteristics and condition of the item. This data is sent to the market price calculation means.

[1681] Step 3:

[1682] The server uses a means for calculating a market price to calculate the market price of the item in the secondary market. The input is data indicating the characteristics and condition of the item, and the output is the market price. This market price is sent to the means for displaying it in augmented reality.

[1683] Step 4:

[1684] The terminal uses a means for displaying in augmented reality to display the calculated market price superimposed on a real image. The input is the market price, and the output is the market price displayed in augmented reality. The user confirms this display.

[1685] Step 5:

[1686] A user puts up an item for sale based on the market price. The input is the user's operation, and the output is the listing information. The listing information is sent to the server.

[1687] Step 6:

[1688] The server uses an emotion recognition means to analyze and recognize the emotion of the user. The input is the user's facial expression and voice data, and the output is the recognized emotion data. This emotion data is sent to the means for recommending items for sale.

[1689] Step 7:

[1690] The server generates a message recommending an item based on the recognized emotion. The input is the emotion data, and the output is the recommendation message. This message is sent to the terminal and displayed to the user.

[1691] Step 8:

[1692] The user checks the recommendation message and modifies the listing information if necessary. The input is the recommendation message and the user's operation, and the output is the final listing information. This listing information is sent to the server, and the product is listed through the API of the online shopping site.

[1693] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1694] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1695] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1696] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1697] [Fourth embodiment]

[1698] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1699] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1700] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1701] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1702] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1703] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1704] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1705] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1706] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1707] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1708] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1709] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1710] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1711] "Example 1"

[1712] In one embodiment of the present invention, a user takes a picture of an object they own using a camera such as a smartphone. The captured image is analyzed by an image analysis means to determine the type and condition of the object. Based on the results of this determination, the market price of the object in secondary distribution is calculated. The calculated market price is displayed to the user using AR (augmented reality) technology. Specifically, when the user looks at an object through the smartphone camera, the market price is displayed above the object.

[1713] "Example 2"

[1714] Furthermore, users can list items based on the displayed market price. Specifically, when a user taps on an item with a market price displayed on the smartphone screen, the listing screen for that item is displayed. On this screen, users can enter the listing price and listing comment, and then press the listing button to list the item.

[1715] The process flow of each embodiment will be described below.

[1716] "Example 1"

[1717] Step 1: The user takes a photo of an object they own using a smartphone or other camera.

[1718] Step 2: The captured image is analyzed by an image analysis means to determine the type and condition of the object.

[1719] Step 3: Based on the results of the assessment, the market price for the item in secondary distribution is calculated.

[1720] Step 4: The calculated price is displayed to the user using augmented reality (AR) technology. Specifically, when the user looks at an object through the smartphone camera, the price is displayed on top of the object.

[1721] "Example 2"

[1722] Step 1: The user taps on an object on their smartphone screen that displays a market price.

[1723] Step 2: When you tap, the listing screen for that item will appear.

[1724] Step 3: On this screen, the user enters the listing price and listing comments.

[1725] Step 4: You can list your item by pressing the list button.

[1726] Example 1

[1727] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1728] With conventional systems, it was difficult for users to quickly and accurately grasp the market price of the items they owned in the secondary distribution market. In addition, there was a lack of a way to display market price information in an easy-to-understand visual format, so users could not fully utilize it as a basis for making decisions when listing items. This made it difficult for users to buy and sell items at the appropriate price, resulting in a problem of reduced efficiency in transactions.

[1729] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1730] In this invention, the server includes a means for a user to photograph an object using a photographing device, a means for transmitting the photographed image to an analysis device, a means for the analysis device to determine the type and condition of the object using image analysis technology, a means for calculating the market price of the object in the secondary distribution market based on the determination result, a means for transmitting the calculated market price to a display device, and a means for the display device to display the market price using augmented reality technology. This allows the user to quickly and accurately grasp the market price of the object they own and to confirm it in a visually easy-to-understand manner. In addition, the user can list the object at an appropriate price, improving the efficiency of transactions.

[1731] A "user" is an individual or corporation that uses the system to photograph items, check market prices, and list them for sale.

[1732] An "imaging device" is a device such as a camera or smartphone that a user uses to photograph an object.

[1733] An "analysis device" is a computer system that receives captured images and uses image analysis technology to determine the type and condition of an object.

[1734] "Image analysis technology" refers to algorithms and software for analyzing image data to determine the type and condition of an object.

[1735] A "secondary market" is a market in which goods are bought and sold again after they have been in the hands of consumers.

[1736] "Market price" refers to the general trading price of a product in the secondary distribution market.

[1737] A "display device" is a device for visually displaying the calculated market price to a user, and includes a smartphone, tablet, etc.

[1738] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[1739] "Listing" refers to the act of a user offering something they own on the market for sale.

[1740] This invention is a system that quickly and accurately grasps the market price in the secondary distribution market of an item owned by a user and displays it in a visually easy-to-understand manner. A specific embodiment of this system is described below.

[1741] A user takes a picture of an object using a camera such as a smartphone. The captured image is sent to an analysis device via a smartphone application. The analysis device uses image analysis technology such as Google Cloud Vision API to determine the type and condition of the object in the image. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch can be determined.

[1742] The analysis device calculates the market price of the item in the secondary distribution market based on the judgment result. Specifically, it refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis result determines that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[1743] The calculated market price is sent from the analysis device to a smartphone. The smartphone displays the received market price using AR (Augmented Reality) technology. Specifically, when a user looks at an object through the smartphone's camera, the market price is displayed above the object. For example, when a user looks at a watch through the smartphone's camera, "Market price: \10,000" is displayed above the watch. Libraries such as ARKit (iOS) and ARCore (Android) are used for AR display.

[1744] This system allows users to quickly and accurately grasp the market price of their items and check it in a visually easy-to-understand format. It also allows users to list their items at an appropriate price, improving the efficiency of transactions.

[1745] As a concrete example, consider the case where a user has an old Seiko watch. The user activates the camera on their smartphone and takes a picture of the watch. The captured image is sent from the smartphone to a server, where the brand, model, and condition of the watch are analyzed using the Google Cloud Vision API. Based on the analysis results, a market price is calculated using the eBay API and past transaction prices of similar watches. The calculated market price is sent to the smartphone, and when the user looks at the watch through the camera, "Market Price: \10,000" is displayed above the watch.

[1746] An example of a prompt sentence might be:

[1747] "Write a program that takes a picture of an old Seiko watch, analyzes the brand, model and condition of the watch using the Google Cloud Vision API, calculates a market price by referencing the eBay API, and uses AR technology to display the market price on top of the watch through the smartphone camera."

[1748] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1749] Step 1:

[1750] The user takes a picture of an object using the smartphone camera.

[1751] Input: An image of an object taken by the user.

[1752] Specific operation: The user launches the smartphone camera app and takes a picture of an object they own. For example, if the user has an old watch, they can take a picture of the watch with the camera. When they press the capture button, the image is saved on the smartphone.

[1753] Output: The captured image data.

[1754] Step 2:

[1755] The terminal transmits the captured image to the server.

[1756] Input: The captured image data.

[1757] Specific operation: A smartphone application uploads image data to a server using an HTTP request. For example, the image data is encoded in Base64 format and sent in the body of a POST request.

[1758] Output: Image data sent to the server.

[1759] Step 3:

[1760] The server performs image analysis.

[1761] Input: Image data sent to the server.

[1762] How it works: The server uses image analysis technology such as the Google Cloud Vision API to determine the type and condition of objects in the image. The server sends image data to the API and receives the analysis results in return. For example, the brand, model, and condition (new, used, damaged, etc.) of a watch are determined.

[1763] Output: Analysis results regarding the type and condition of the item.

[1764] Step 4:

[1765] The server calculates the market price of the item in the secondary distribution market.

[1766] Input: Analysis results regarding the type and condition of the item.

[1767] Specific operation: Based on the analysis results, the server refers to databases such as eBay API and Amazon API, and calculates the market price based on the past transaction prices of similar items. For example, if the analysis results determine that the item is a "second-hand Seiko watch," it uses the eBay API to obtain the transaction prices of similar second-hand Seiko watches and calculates the average price.

[1768] Output: The calculated market price.

[1769] Step 5:

[1770] The server transmits the market price to the terminal.

[1771] Input: The calculated market price.

[1772] Specific operation: The server generates JSON data including the market price and returns it to the terminal as an HTTP response. For example, it returns data in the format {"item": "Seiko Watch", "price": "\10,000"}.

[1773] Output: Quote price data sent to the terminal.

[1774] Step 6:

[1775] The device displays market prices using AR technology.

[1776] Input: Quote price data sent to the terminal.

[1777] Specific operation: The smartphone uses AR (Augmented Reality) technology to display the received market price. Specifically, the market price is overlaid on the user's smartphone camera view. For example, when the user looks at their watch through the camera, "Market price: ¥10,000" is displayed on top of the watch. Libraries such as ARKit (iOS) and ARCore (Android) are used for the AR display.

[1778] Output: Quoted prices displayed using AR technology.

[1779] (Application example 1)

[1780] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1781] With conventional systems, it was difficult for users to easily grasp the market price of their items in the secondary market. In addition, in order to check the market price, users had to use multiple websites and applications, which was time-consuming. Furthermore, there was a lack of a way to check the market price in real time when appraising or purchasing items in a physical store.

[1782] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating the market price of the item in the secondary distribution based on the image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing the item based on the displayed market price, a means for photographing the item using the camera of the smart device and determining the type and condition of the item using image analysis technology, a means for acquiring market price data from a database based on the determination result and calculating the market price, a means for displaying the calculated market price on the screen of the smart device using AR technology, and a means for generating a prompt sentence for calculating the market price using a generative AI model. This allows the user to easily check the market price of the item using the smart device and grasp the market price in real time when the item is appraised or purchased in a physical store.

[1783] "Image analysis means" is a technology for analyzing image data acquired through a camera and determining the type and condition of an object.

[1784] The "market price calculation means" is a technology for calculating the market price of an item in the secondary distribution market based on the data obtained by the image analysis means.

[1785] "Means of displaying using AR (Augmented Reality)" refers to technology for overlaying the calculated market price on the screen of a smart device.

[1786] A "means of listing" refers to technology that allows an item to be listed on an online marketplace, etc., based on user operations.

[1787] "Smart device" is a general term for portable electronic devices such as smartphones and tablets that can connect to the Internet and have a camera function.

[1788] "Image analysis technology" is a technology for analyzing image data captured by a camera and extracting specific information.

[1789] "Database" means an information system that stores market data and other related information in an organized manner and enables it to be searched and retrieved as needed.

[1790] A “generative AI model” is a model that is trained to perform a specific task using artificial intelligence, such as in this case to generate prompts for calculating quotes.

[1791] A "prompt sentence" is an input sentence that causes a generative AI model to perform a specific task.

[1792] A system for implementing the present invention uses the following hardware and software.

[1793] Hardware used

[1794] Smart devices (smartphones, tablets, etc.)

[1795] Server (image analysis, database)

[1796] Software used

[1797] Image analysis library (e.g. OpenCV)

[1798] AR library (e.g. ARCore)

[1799] Database (e.g. MySQL)

[1800] Generative AI models (e.g. GPT-4)

[1801] System configuration and operation

[1802] 1. Image capture

[1803] The user uses the camera on their smart device to take a picture of the item they want to appraise, and the image data is temporarily stored on the smart device.

[1804] 2. Image Analysis

[1805] The smart device sends the captured image data to the server, which uses an image analysis library (e.g., OpenCV) to analyze the image data and determine the type and condition of the product.

[1806] 3. Market Calculation

[1807] Based on the results of image analysis, the server retrieves market price data for the relevant item in the secondary distribution market from a database (e.g. MySQL), and calculates the market price based on the retrieved data.

[1808] 4. AR display

[1809] The calculated market price is sent from the server to the smart device, which uses an AR library (e.g., ARCore) to display the market price overlaid on the product seen through the camera.

[1810] 5. Use of generative AI models

[1811] The server uses a generative AI model (e.g. GPT-4) to generate a prompt to calculate the market price. The generated prompt has the following format:

[1812] "What type of item is this, and what is the current going price on the secondary market?"

[1813] Examples

[1814] For example, consider the case where a user wants to purchase a used smartphone at a physical store. The user takes a picture of the product using the smartphone camera. The captured image is sent to the server and analyzed using OpenCV. Based on the analysis results, the market price of the same model in the secondary market is obtained from the database. The calculated market price is displayed on the smartphone screen using ARCore. In this way, the user can easily check the market price when purchasing the product at a physical store.

[1815] This system allows users to easily check the market price of items using their smart devices, and understand the real-time market price when having items appraised or purchased in a physical store.

[1816] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1817] Step 1:

[1818] The user uses the camera of the smart device to take a picture of the item to be appraised. The input is the product image, and the output is the image data temporarily stored in the smart device. In concrete terms, the user starts the camera app, frames the product, and presses the shutter button.

[1819] Step 2:

[1820] The terminal transmits the captured image data to the server. The input is the temporarily stored image data, and the output is the image data transmitted to the server. In concrete terms, the terminal uploads the image data to the server via an Internet connection.

[1821] Step 3:

[1822] The server uses an image analysis library (e.g. OpenCV) to analyze the image data and determine the type and condition of the product. The input is the image data sent to the server, and the output is data about the type and condition of the product. Specifically, the server runs an image analysis algorithm to extract features in the image and identify the product.

[1823] Step 4:

[1824] Based on the results of image analysis, the server retrieves secondary market price data for the relevant product from a database (e.g. MySQL). The input is data about the type and condition of the product, and the output is price data. Specifically, the server executes a database query to search for price information for the relevant product.

[1825] Step 5:

[1826] The server calculates the market price based on the acquired market price data. The input is the market price data, and the output is the calculated market price. Specifically, the server uses statistical methods and algorithms to analyze the market price data and calculate the average price and median price.

[1827] Step 6:

[1828] The server transmits the calculated market price to the smart device. The input is the calculated market price, and the output is the market price transmitted to the smart device. In particular, the server transmits market price data to the smart device via an internet connection.

[1829] Step 7:

[1830] The device uses an AR library (e.g. ARCore) to overlay the market price on the product seen through the camera. The input is the market price sent from the server, and the output is the market price displayed on the smart device screen. In concrete terms, the device overlays the market price on the camera image.

[1831] Step 8:

[1832] The server uses a generative AI model (e.g. GPT-4) to generate a prompt for calculating the market price. The input is data on the type and condition of the product, and the output is the generated prompt. In concrete terms, the server requests the generative AI model to generate a prompt, and obtains the following prompt:

[1833] "What type of item is this, and what is the current going price on the secondary market?"

[1834] In this way, users can easily check the market price of items using their smart devices and understand the market price in real time when having an item appraised or purchased in a physical store.

[1835] Example 2

[1836] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1837] With conventional systems, it was difficult to accurately calculate the market price of goods in the secondary distribution market and provide an environment where users could easily list items for sale. In addition, there was a lack of a way to effectively utilize past transaction data when calculating market prices, so it was not possible to provide users with reliable market price information. Furthermore, the listing process was complicated, placing a heavy burden on users.

[1838] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an image analysis means, a means for calculating a market price in the secondary distribution of an item based on image data obtained by the image analysis means, a means for calculating a market price using a generative AI model, a means for displaying the calculated market price, a means for listing an item based on the displayed market price, a means for inputting listing information based on a user's operation, a means for transmitting the input listing information to the server, and a means for storing the listing information received by the server in a database. This enables a user to easily list an item based on highly reliable market price information.

[1839] An "image analysis means" is a device or software that analyzes image data acquired through a camera and recognizes the characteristics and condition of an object.

[1840] The "market price calculation means" refers to a device or software for calculating the price of an item in the secondary market based on the data obtained by the image analysis means and past transaction data.

[1841] A "generative AI model" is a model that uses artificial intelligence techniques to analyze data and perform specific tasks, in this case to calculate quotes.

[1842] A "means for displaying quotes" is a device or software for visually presenting the calculated quotes to a user.

[1843] A "means for listing" is a device or software that supports a series of operations for a user to list an item on the secondary market.

[1844] "Means for inputting listing information" refers to a device or software that allows a user to input information such as the price and comments of the item being listed.

[1845] The "means for transmitting the selling information to the server" refers to a device or software for transmitting the selling information input by the user to the server.

[1846] The "means for storing in a database" refers to a device or software that stores the listing information received by the server in a database and makes it available for other users to view.

[1847] This invention is a system that allows a user to list items for sale using a smartphone, and includes an image analysis means, a means for calculating a market price, a generative AI model, a means for displaying a market price, a means for listing an item, a means for inputting listing information, a means for transmitting the listing information to a server, and a means for saving the listing information in a database.

[1848] The user starts the app on their smartphone and uses the camera to capture an image of the item they wish to sell. The image analysis means analyzes the captured image data and recognizes the characteristics and condition of the item. The analyzed data is then sent to the server.

[1849] The server retrieves historical trading data from a database (e.g., MySQL) and calculates quotes using a generative AI model (e.g., OpenAI's GPT-4). For example, it sends a prompt like the following to the generative AI model:

[1850] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[1851] Transaction 1: 400 yen

[1852] Transaction 2: 500 yen

[1853] Transaction 3: 600 yen

[1854] The generative AI model calculates the market price based on this data, and the server sends the calculated market price to the user's device.

[1855] The terminal displays the received market price on the screen. For example, it may say, "The market price for this book is 500 yen." The user selects the item to sell based on the displayed market price, and inputs the selling price and listing comment on the listing screen. For example, the user sets the selling price to 600 yen and inputs the comment, "The item is in good condition."

[1856] When the user presses the sell button, the terminal sends the entered selling information to the server, which then stores the received selling information in a database and makes it available for other users to view.

[1857] The system allows users to easily list items based on reliable market information, and by using generative AI models, it effectively leverages past transaction data to provide accurate market prices.

[1858] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1859] Step 1:

[1860] The user launches the app.

[1861] A user launches an app by tapping the app icon on the home screen of their smartphone. The input is the user's operation, and the output is the launch of the app.

[1862] Step 2:

[1863] A user uses a camera to capture an image of an object.

[1864] A user uses the camera function in the app to take a picture of the item they want to put up for sale. The input is the image data acquired through the camera, and the output is the image data sent to the image analysis means.

[1865] Step 3:

[1866] The server analyzes the image data.

[1867] The server uses image analysis means to analyze the acquired image data and recognize the features and state of the object. The input is the image data sent by the user, and the output is the analyzed feature data of the object.

[1868] Step 4:

[1869] The server calculates the market price.

[1870] The server retrieves historical trading data from the database and uses the generative AI model to calculate quotes. For example, it sends the following prompt to the generative AI model:

[1871] "Calculate the market price of an old book based on historical transaction data. Here is the data:

[1872] Transaction 1: 400 yen

[1873] Transaction 2: 500 yen

[1874] Transaction 3: 600 yen

[1875] The input is historical trading data and a prompt, and the output is the calculated quote price.

[1876] Step 5:

[1877] The server transmits the market price to the terminal.

[1878] The server sends the calculated quotes to the user's terminal. The input is the calculated quote and the output is the quote sent to the terminal.

[1879] Step 6:

[1880] The terminal will display the market price.

[1881] The terminal displays the received market price on the screen. For example, it displays "The market price of this book is 500 yen." The input is the market price sent from the server, and the output is the market price displayed on the screen.

[1882] Step 7:

[1883] The user selects the item to sell.

[1884] The user taps the item they want to sell from the list of market prices displayed on the screen. The input is the user's operation, and the output is a detailed screen of the selected item.

[1885] Step 8:

[1886] The device displays the listing screen.

[1887] The terminal opens a listing screen that displays detailed information about the selected item. The input is the information about the item selected by the user, and the output is the listing screen.

[1888] Step 9:

[1889] The user enters listing information.

[1890] The user inputs the listing price and listing comment on the listing screen. For example, the listing price is set to 600 yen and the comment "Condition is good." The input is the listing information entered by the user, and the output is the entered listing information.

[1891] Step 10:

[1892] The user presses the sell button.

[1893] The user presses the "list" button to confirm the information they have entered. The input is the user's operation, and the output is the confirmed listing information.

[1894] Step 11:

[1895] The terminal transmits the listing information to the server.

[1896] The terminal transmits the inputted selling information to the server. The input is the confirmed selling information, and the output is the selling information transmitted to the server.

[1897] Step 12:

[1898] The server stores the listing information in a database.

[1899] The server stores the received listings in a database and makes them available for other users to view. The input is the listing sent to the server and the output is the listing stored in the database.

[1900] (Application example 2)

[1901] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1902] In the conventional secondary distribution of goods, it was difficult for users to grasp the appropriate market price, and the listing process was cumbersome. In addition, it was time-consuming for users to input the listing price and comments, and the listing information was not saved efficiently. This reduced user convenience and hindered the revitalization of the secondary distribution market.

[1903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1904] In this invention, the server includes an image analysis means, a means for calculating a market price for an item in secondary distribution based on image data obtained by the image analysis means, a means for displaying the calculated market price in AR (Augmented Reality), a means for listing an item based on the displayed market price, and a mea...

Claims

1. A means for acquiring image data including an image of an article transmitted from a terminal device; means for identifying the type, brand, model and condition of said item by analyzing said image data; A means for calculating a transaction price of the item in a secondary market based on the identified type, brand, model and condition of the item; A means for transmitting the calculated transaction price to the terminal device; A system including:

2. The calculated transaction price is displayed on the screen of the terminal device, superimposed on the image of the item. The system of claim 1 .

3. The image data is generated by a camera provided in the terminal device. The system of claim 1 .

Images