System

Abstract

A system is provided.SOLUTION: A system comprising: means for attaching position information to a photographed image; means for uploading the image and the position information to a server; means for generating useful information based on the image and the position information in the server; means for setting a selling price of the generated information; means for storing the generated information in a database so that the information can be retrieved; means for retrieving the information from the database and displaying a result; means for purchasing the retrieved information and giving a reward to an information provider when a purchase procedure is completed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional information provision systems have the problem that users who need the information cannot search for it unless the information provider properly describes the keywords. Another issue is that it is currently difficult for information providers to receive information fees. Furthermore, while there is a demand for generating useful information in real time and providing it to users who need it, no specific method for achieving this has been established. [Means for solving the problem]

[0005] The present invention solves the above problem by providing a system including: means for assigning location information to a captured image; means for uploading the image and location information to a server; means for generating useful information on the server based on the image and location information; means for setting a selling price for the generated information; means for saving the generated information in a database and making it searchable; means for searching for information from the database and displaying the results; and means for purchasing the searched information and, once the purchase procedure is completed, for rewarding the information provider.

[0006] "Images" refers to photographs and video data.

[0007] "Location Information" refers to geographical information such as latitude and longitude obtained using technologies such as GPS.

[0008] "Means" refers to functions or modules designed to achieve a specific purpose.

[0009] "Server" refers to a computer system that stores, processes, and distributes data over a network.

[0010] "Generative AI" refers to algorithms that use artificial intelligence technology to automatically analyze and generate useful information from input data.

[0011] "Database" refers to an information management system for systematically storing and managing information.

[0012] "Search" refers to the operation of locating information stored in a database based on specific conditions.

[0013] "Information provider" refers to a user who uploads images and location information to the server.

[0014] "Information Searcher" refers to a user who uses the system to locate specific information.

[0015] "Sales pricing means" refers to a feature or module designed to determine a price based on the value of the information generated.

[0016] A means of providing "rewards" refers to a function or module designed to return benefits to information providers in the form of money, points, etc.

[0017] "Useful information" refers to the results of analysis by generative AI and data that is valuable to users. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system that automatically generates useful information using AI based on images and location information taken by users, and sells that information. Specific embodiments for implementing the present invention are described below.

[0040] User capture and upload of images and location information

[0041] Users use mobile devices such as smartphones and tablets to take photos (or videos) at specific locations. For example, users may take photos at tourist spots, shopping malls, parking lots, etc. When this happens, the device automatically uses its GPS function to obtain location information (latitude and longitude).

[0042] After taking a photo, the user uploads the photo and location information to the server using a dedicated application. The user's operation is extremely simple, and the data can be sent simply by following the application's instructions.

[0043] Server-based data processing and information generation

[0044] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[0045] Road congestion

[0046] Congestion levels at stores and tourist spots

[0047] Parking availability

[0048] Real-time weather information

[0049] Flowering status

[0050] For example, the generative AI can analyze the number of cars and the density of people in an image to determine the congestion level of roads and tourist spots. It can also determine the availability of parking spaces based on the usage status of parking spaces visible in the image. It can also obtain weather information that can be determined from the image.

[0051] The information generated by the AI ​​is returned to the server, which stores it in a database. At the same time, the system also includes a function to automatically set a selling price based on the scarcity and demand of the information.

[0052] Searching for and purchasing information

[0053] When a user wants to search for specific information, they specify the location they want to know about using the application's map interface. For example, if a user wants to know the crowd situation at a particular tourist spot, they specify that point on the map. The server then searches the database for information registered near the specified point and displays the results to the user.

[0054] The user selects the information they need from the displayed results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[0055] Specific examples

[0056] Example 1: Providing information on the congestion status of tourist spots

[0057] 1. User: Takes and uploads a photo at tourist spot A.

[0058] 2. Server: Receives images and location information and passes them to the generation AI.

[0059] 3. Generative AI: Analyzes the level of congestion at tourist spot A and returns the information to the server.

[0060] 4. Server: Sets the selling price of the information and stores it in the database.

[0061] If another user wants to know the congestion situation at tourist spot A,

[0062] 1. User: Specify tourist spot A via the map interface.

[0063] 2. Server: Search and display information about tourist spot A.

[0064] 3. User: Purchases information and obtains congestion status provided by the server.

[0065] Example 2: Providing parking availability information

[0066] 1. User: Takes and uploads a photo in the parking lot.

[0067] 2. Server: Receives images and location information and passes them to the generation AI.

[0068] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[0069] 4. Server: Sets the selling price of the information and stores it in the database.

[0070] If another user wants to know the availability of that parking space,

[0071] 1. User: Specify parking via map interface.

[0072] 2. Server: Searches and displays information about nearby parking lots.

[0073] 3. User: Purchases information and obtains availability provided by the server.

[0074] These embodiments allow users to easily obtain the information they need and provide benefits to information providers. The system provides an efficient means for widely sharing valuable information in real time.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] User: Uses a smartphone or tablet to take images (photos or videos) at specific locations, such as tourist spots or parking lots.

[0078] Step 2:

[0079] Device: When an image is taken, the location information (latitude and longitude) is automatically obtained using the GPS function.

[0080] Step 3:

[0081] User: Uploads the captured images and their location information to the server via a dedicated application.

[0082] Step 4:

[0083] Server: Receives images and location information uploaded by users.

[0084] Step 5:

[0085] Server: Passes the received data to the generation AI.

[0086] Step 6:

[0087] Generative AI: Using image analysis algorithms to analyze elements within an image and generate useful information such as:

[0088] Identify road congestion based on the number of cars and people.

[0089] Analyzes the density of people to determine how crowded stores and tourist spots are.

[0090] To identify parking availability based on parking space usage.

[0091] Determine the weather (sunny, cloudy, rainy, etc.) from the photo.

[0092] Analyze the type of flower and its flowering status.

[0093] Step 7:

[0094] Generative AI: Returns the generated useful information to the server.

[0095] Step 8:

[0096] Server: Sets a selling price based on the generated information, taking into account factors such as scarcity and demand for the information.

[0097] Step 9:

[0098] Server: Stores the generated information in a database.

[0099] Step 10:

[0100] Server: Makes the stored information searchable.

[0101] Step 11:

[0102] User (information seeker): Uses the application's map interface to search for information by specifying the location of interest.

[0103] Step 12:

[0104] Server: Searches for information stored in a database near a specified location.

[0105] Step 13:

[0106] Server: Displays the information to the user as search results.

[0107] Step 14:

[0108] User: Select the necessary information from the search results and proceed with the purchase.

[0109] Step 15:

[0110] Server: Provides the user with the relevant information once the purchase is complete.

[0111] Step 16:

[0112] Server: Determines the reward for the informant and reflects it in the account.

[0113] In this way, the system allows users to easily obtain the information they need through specific processing steps, and also allows information providers to easily obtain profits.

[0114] Example 1

[0115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] In modern society, there is a growing demand for useful information obtained in real time. However, existing systems have limitations in the means to efficiently collect, analyze, and provide such information. In particular, systems that combine images and location information to provide information intuitively and immediately have not yet been fully developed. Therefore, an objective of the present invention is to provide a system that automatically generates useful information in real time using images and location information taken by users and makes that information available for sale.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0118] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to a data storage, means for generating useful information in the data storage based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a search system and making it searchable, means for searching for information from the search system and displaying the results, and means for purchasing the searched information and providing a reward to the information provider when the purchase procedure is completed. This allows users to easily obtain useful information in real time, and enables information providers to easily earn profits.

[0119] "Means for adding location information to captured images" refers to a function that uses the camera and GPS functions built into the user's mobile device to automatically add location information such as latitude and longitude to images when they are taken.

[0120] The "means for uploading the images and location information to data storage" refers to a function that allows a user to send the captured images and their location information to a server via a dedicated application and store them in cloud-based data storage.

[0121] The "means for generating useful information based on the images and location information in the data storage" refers to a function that uses a generation AI model in the server to analyze the received images and location information and generate useful information (such as congestion status and parking availability).

[0122] The "means for setting the selling price of the generated information" is an algorithm for evaluating the value of the information generated by the server and automatically calculating an appropriate selling price based on the rarity and demand of the information.

[0123] "Means for storing the generated information in a search system and making it searchable" refers to a function that stores the generated useful information in a database so that users can easily find it through a search system.

[0124] "Means for searching for information from the search system and displaying the results" refers to a function in which, when a user enters specific information into the search system, the server searches for the corresponding information from the database and displays the results on the user's terminal.

[0125] "Means for purchasing searched information and rewarding the information provider once the purchase procedure is completed" refers to a system that manages the payment process for purchasing information found by a user and pays a reward to the user who provided the information once the purchase is completed.

[0126] "Traffic congestion status" refers to information about the density of vehicles and the degree of congestion on specific roads, intersections, etc.

[0127] "Crowding levels at hubs and tourist destinations" refers to information about the density of people and the degree of congestion at specific facilities or areas.

[0128] "Parking area availability" is information about the number and location of available parking spaces in a particular parking lot.

[0129] "Real-time weather information" refers to information about current weather conditions, such as temperature and precipitation, in a particular area.

[0130] "Flowering status of plants" is information about the progress of flowering and the best time to see a particular plant.

[0131] A "generative AI model" is an artificial intelligence algorithm with functions such as image analysis and data generation, and in particular uses computer vision and natural language processing to generate useful information.

[0132] This invention is a system that automatically generates useful information using a generative AI model based on images taken by users and location information, and sells that information. Specific embodiments for implementing the invention are described below.

[0133] Overall system configuration

[0134] Users take photos and videos at specific locations using mobile devices such as smartphones and tablets. At this time, the device uses its GPS function to obtain location information (latitude and longitude). After taking the photos, users upload the images and location information to a server using a dedicated application.

[0135] The server receives images and location information uploaded by users and automatically generates useful information based on that information using a generative AI model. The generated information is stored in a database and made searchable, with a selling price automatically set based on the information's rarity and demand. Finally, when the user searches for and purchases the information and the purchase process is completed, the information provider is rewarded.

[0136] Hardware and software used

[0137] Hardware

[0138] Smartphones and tablets (e.g., iPhone (registered trademark), ANDROID (registered trademark) smartphones, iPad (registered trademark))

[0139] Cloud server (e.g., AWS (registered trademark), Google (registered trademark), Microsoft (registered trademark) Azure (registered trademark))

[0140] software

[0141] Camera app and GPS function on mobile devices

[0142] Dedicated app (iOS app, Android app)

[0143] Generative AI models (e.g., OpenAI® GPT-4®)

[0144] Database management system (e.g., MySQL (registered trademark), PostgreSQL)

[0145] Specific details of data processing and calculation

[0146] Images taken by users on their mobile devices and the acquired location information are uploaded to the server via a dedicated app. The server then performs the following processes.

[0147] 1. Data integrity check

[0148] The server checks the consistency of the received image data and location information, checking for errors and omissions.

[0149] 2. Data Preprocessing

[0150] The server converts the image data into a format that can be passed to the generative AI model (e.g., JSON format).

[0151] 3. Analysis and information generation using generative AI models

[0152] The generative AI model analyzes features in an image (e.g., number of cars, human density, weather, etc.) and generates useful information (e.g., congestion level, parking availability, etc.).

[0153] Example of a prompt to be passed to the AI ​​generator:

[0154] "Please analyze the congestion situation at store X from this image."

[0155] "Please identify the availability of this parking space from this image."

[0156] 4. Information Storage and Pricing

[0157] The server stores the generated information in a database and makes it searchable.

[0158] The server automatically sets the selling price based on the scarcity and demand of the information.

[0159] Specific examples

[0160] Example 1: Providing information on the congestion status of tourist spots

[0161] 1. The user takes a photo at tourist spot A and uploads it to the server via a dedicated app.

[0162] 2. The server receives the image and location information and passes it to the generative AI model.

[0163] 3. The generative AI model analyzes the level of congestion at tourist destination A and returns the results to the server.

[0164] 4. The server sets the selling price of the information and stores it in the database.

[0165] If another user wants to know the congestion situation at tourist spot A,

[0166] 1. The user specifies tourist spot A via the map interface.

[0167] 2. The server searches for and displays information about tourist spot A.

[0168] 3. The user purchases information and obtains congestion status provided by the server.

[0169] Example 2: Providing parking availability information

[0170] 1. The user takes a photo in the parking lot and uploads it to the server via a dedicated app.

[0171] 2. The server receives the image and location information and passes it to the generative AI model.

[0172] 3. The generative AI model analyzes the parking availability and returns the results to the server.

[0173] 4. The server sets the selling price of the information and stores it in the database.

[0174] If another user wants to know the availability of that parking space,

[0175] 1. The user specifies a parking spot via the map interface.

[0176] 2. The server searches for and displays information about nearby parking lots.

[0177] 3. The user purchases the information and obtains the availability provided by the server.

[0178] As described above, this system allows users to easily obtain useful information in real time, and information providers can easily benefit from it. This system provides an efficient means for widely sharing valuable information.

[0179] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0180] Step 1:

[0181] User capture and upload of images and location information

[0182] Operation overview

[0183] Input: Images taken by the user on a mobile device and acquired location information

[0184] How it works: The user launches the dedicated application and takes a photo or video at a specific location. At this time, the location information (latitude and longitude) is automatically obtained using the mobile device's GPS function. After taking the photo, the user presses the send button in the dedicated application to upload the image and location information to the server.

[0185] Output: Image data and location information sent to the server

[0186] Step 2:

[0187] Data reception and preprocessing by the server

[0188] Operation overview

[0189] Input: Image data and location information uploaded by the user

[0190] How it works: The server receives an HTTP request and retrieves image and location data. The server checks the data for integrity and errors or omissions. It then converts the image data into an appropriate format (e.g., JSON) to be passed to the generative AI model.

[0191] Output: Preprocessed image data and location information

[0192] Step 3:

[0193] Information generation by generative AI

[0194] Operation overview

[0195] Input: Preprocessed image data and location information

[0196] How it works: The server sends the preprocessed data to a generative AI model. The generative AI model analyzes features in the image (e.g., number of cars, human density, weather, etc.) and automatically generates useful information (e.g., congestion level, parking availability, etc.). Examples of prompts input to the generative AI model include "Please analyze the congestion level at Store X from this image" and "Please identify the availability of this parking lot from this image."

[0197] Output: Useful information as analysis results (such as congestion status and parking availability)

[0198] Step 4:

[0199] Server-based information storage and pricing

[0200] Operation overview

[0201] Input: Useful information from the analysis results obtained from the generative AI model

[0202] How it works: The server stores the generated information in a database. It then applies an algorithm to automatically set the selling price based on the scarcity and demand of the information, and determines an appropriate selling price. The information, along with the selling price, is made searchable.

[0203] Output: Useful information stored in the database and its selling price

[0204] Step 5:

[0205] User search and purchase

[0206] Operation overview

[0207] Input: Search criteria specified by the user (e.g., the congestion status of a specific location)

[0208] How it works: The user specifies the location they want to know about using the map interface in the dedicated app. The server searches the database for information close to the specified location and displays the results to the user. The user selects the information they need from the displayed information and completes the purchase procedure. Once payment is complete, the server provides the information to the user and simultaneously rewards the information provider.

[0209] Output: Useful information provided to the user and the reward given.

[0210] (Application example 1)

[0211] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0212] In modern urban environments, safety management of public areas and private property is a critical issue. In particular, during large-scale events and disasters, it is necessary to grasp safety situations in real time and respond quickly, but current systems are insufficient in this regard. Furthermore, there is no established method for efficiently collecting this safety information and appropriately rewarding data providers. Furthermore, there is a lack of ways for ordinary users to contribute to public safety using their own mobile devices.

[0213] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0214] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for evaluating the safety of a specific area based on the generated information and issuing an alert. This allows users to easily collect and provide safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[0215] output

[0216] The "means for assigning location information to a captured image" refers to a device or software that provides a function for automatically associating location information such as latitude and longitude with an image captured by a user.

[0217] The "means for uploading the images and location information to the server" refers to a device or software that includes a network connection and communication protocol for transmitting images taken by a user and their associated location information to the server.

[0218] The "means for generating useful information based on the images and location information at the server" refers to an algorithm or computer program that analyzes the received images and location information and generates important data or insights based on specific conditions.

[0219] The "means for setting the selling price of the generated information" is an algorithm or computer program that determines an appropriate selling price based on the market value and demand for the generated useful information.

[0220] "Means for storing the generated information in a database and making it searchable" refers to a database management system that efficiently stores the generated information and makes it easily searchable and accessible later.

[0221] The "means for searching information from the database and displaying the results" refers to an interface and algorithms for extracting relevant information from the database based on a search query from a user and displaying the results to the user.

[0222] "Means for purchasing searched information and rewarding the information provider when the purchase procedure is completed" is a system that automatically awards money or points to the user who provided the information when the transaction is completed after the user purchases the information.

[0223] The "means for assessing the safety of a specific area based on the generated information and issuing an alert" refers to an algorithm and communication device for analyzing the generated information, assessing the safety of a specific area, and issuing an alert to relevant parties as necessary.

[0224] output

[0225] This invention is a system that uses AI to automatically generate useful information based on images and location information taken by users, and provides and sells that information. This system is particularly designed for application in the field of safety management, and is capable of issuing area safety assessments and warnings in real time.

[0226] 1. Acquisition and upload of images and location information by user devices

[0227] Users use mobile devices such as smartphones and tablets to take photos at specific locations. For example, users may take photos in public areas or on private property. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the photo, users upload the photo and location information to a server using a dedicated application. This allows users to easily provide the necessary data to the system.

[0228] 2. Data processing and information generation by the server

[0229] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[0230] Safety assessment of specific areas (detection of suspicious objects, determination of dangerous areas, etc.)

[0231] Real-time information related to public safety (such as congestion and traffic conditions)

[0232] For example, the generating AI analyzes whether an image contains any suspicious objects and uses that information to evaluate the safety of the area. It also evaluates the safety of the surrounding environment in real time based on data obtained from location information. The generated information is returned to the server, which stores it in a database. At the same time, it automatically sets a selling price based on the scarcity and demand of the information.

[0233] 3. Searching for and Purchasing Information

[0234] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. For example, if a user wants to know the safety status of a particular area, they specify that location on the map. The server searches the database for information registered near the specified location and displays the results to the user. The user selects the information they need from the displayed results and proceeds with the purchase. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[0235] Hardware and software used

[0236] Hardware: Smartphones, tablets

[0237] Software: Dedicated application, server, generative AI, Google Cloud Vision API, Google Maps API, requests library, database management system

[0238] Specific examples

[0239] Example 1: Public Area Safety Assessment

[0240] 1. User: Finds a suspicious object in the park, takes a photo, and uploads it. The location information is automatically captured.

[0241] 2. Server: Receives images and location information and passes them to the generation AI.

[0242] 3. Generative AI: Performs a safety assessment of the park and determines that it is a dangerous area.

[0243] 4. Server: Stores the safety assessment results in a database and issues alerts as necessary.

[0244] Prompt Sentence Examples

[0245] "Generate the following information based on the photo and GPS location:

[0246] Is there anything suspicious in the image?

[0247] Assessment of local crime prevention and safety conditions

[0248] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0249] output

[0250] Step 1:

[0251] A user takes a photo at a specific location using a device such as a smartphone or tablet. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). The input is the captured image and its location information. As a result, the image taken by the user is saved along with the location information.

[0252] Step 2:

[0253] After taking a photo, the user uses a dedicated application to upload the photo and location information to the server. The input is the image and location information acquired in step 1. The output is the image and location information stored on the server. The server receives this information and prepares it for the next processing step.

[0254] Step 3:

[0255] The server passes the received image and location information to the generation AI. The input is the image and location information stored on the server. The generation AI uses image analysis algorithms to detect suspicious objects in the uploaded image and evaluate the safety of the area. This analysis is performed by detecting specific objects in the image and matching them with location information. The output is the results of the image analysis (e.g., whether or not there is a suspicious object, the safety evaluation of the area, etc.).

[0256] Step 4:

[0257] The server sets the selling price of the generated information based on the analysis results received from the generation AI. The input is the image analysis results. This automatically determines an appropriate selling price based on market value and demand. The output is the set selling price.

[0258] Step 5:

[0259] The server stores the generated information and its selling price in a database, making it searchable. The input is the generated information and the set selling price. This allows the database to be searchable and accessible by users later, and the output is the information stored in the database.

[0260] Step 6:

[0261] When a user searches for specific information, they use the application's map interface to specify the location they want to know about. The input is the user's search query (e.g., safety status in a particular area). The server searches a database for relevant information based on the query and displays the results. The output is the displayed search results.

[0262] Step 7:

[0263] The user selects the information they need from the displayed search results and completes the purchase process. The input is the selected information and the purchase process information. Once the purchase process is completed, the server provides the information to the user and awards a reward to the information provider. This allows the provider to receive a reward for providing the information, and the user can obtain the information they need. The output is the information provided to the user and the information provider with the award.

[0264] The above processing steps enable users to collect and provide useful safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[0265] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0266] The present invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for implementing the present invention will be described below.

[0267] User capture and upload of images and location information

[0268] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[0269] Server-based data processing and information generation

[0270] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[0271] Road congestion

[0272] Congestion levels at stores and tourist spots

[0273] Parking availability

[0274] Real-time weather information

[0275] Flowering status

[0276] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[0277] Recognizing user emotions with an emotion engine

[0278] The present invention further incorporates an emotion engine. The emotion engine uses technology to recognize a user's emotion by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data contained in images uploaded by the user to identify the user's emotion (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation method of the generated useful information.

[0279] Searching for and purchasing information

[0280] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[0281] Purchase and Use of Information

[0282] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[0283] Specific examples

[0284] Example 1: Combining tourist destination congestion and user sentiment

[0285] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[0286] 2. Server: Receives images and location information and passes them to the generation AI.

[0287] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[0288] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[0289] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[0290] Example 2: Combining parking availability and user sentiment

[0291] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[0292] 2. Server: Receives images and location information and passes them to the generation AI.

[0293] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[0294] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[0295] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[0296] This allows users to easily obtain the information they need in a form optimized for their emotions, and information providers can easily earn profits. This system provides an efficient means for widely sharing valuable information in real time.

[0297] The processing flow will be explained below.

[0298] Step 1:

[0299] User: Take an image (photo or video) with a smartphone or tablet at a specific location (e.g., a tourist spot or parking lot).

[0300] Step 2:

[0301] Device: The location information (latitude and longitude) of the captured image is automatically obtained using the GPS function.

[0302] Step 3:

[0303] User: The captured images and location information are uploaded to the server through a dedicated application. The application has a user-friendly interface and is easy to operate.

[0304] Step 4:

[0305] Server: Receives images and location information uploaded by users and temporarily stores them in a database.

[0306] Step 5:

[0307] Server: Passes the received image and location information to the generation AI, which runs an image analysis algorithm to analyze elements within the image.

[0308] Step 6:

[0309] Generative AI: Analyzes elements in an image and generates useful information. Examples include:

[0310] Analyze the number of cars and people to identify road congestion.

[0311] Analyzes the density of people to determine how crowded stores and tourist spots are.

[0312] Analyze parking space usage to identify available parking spaces.

[0313] Determine the weather (sunny, cloudy, rainy, etc.) from the image.

[0314] Analyze the type of flower and its flowering status.

[0315] Step 7:

[0316] Generative AI: Returns the generated useful information to the server.

[0317] Step 8:

[0318] Server: Sets a selling price based on the useful information received, taking into account factors such as scarcity and demand for the information.

[0319] Step 9:

[0320] Server: Passes the generated information to the emotion engine and provides the information necessary for analyzing user emotions.

[0321] Step 10:

[0322] Emotion engine: Analyzes image and text content to recognize the user's emotional state (e.g., joy, sadness, surprise, etc.).

[0323] Step 11:

[0324] Emotion engine: Optimizes the presentation of generated information based on the user's emotional state, for example, suggesting less congested routes for users who are feeling stressed.

[0325] Step 12:

[0326] Server: Based on the results of the emotion engine, the optimized information is stored in a database and made searchable.

[0327] Step 13:

[0328] User (information searcher): Uses the map interface of the application to search for information by specifying the location they want to know about. For example, if a user wants to know the congestion status of a tourist spot, they specify that location on the map.

[0329] Step 14:

[0330] Server: Searches the database for registered information near the specified location and displays the results to the user.

[0331] Step 15:

[0332] User: Selects the information they need from the search results and completes the purchase. They also check the information presented based on their emotional state.

[0333] Step 16:

[0334] Server: Once the purchase process is complete, the server provides the relevant information to the user and simultaneously initiates the reward process for the information provider.

[0335] Step 17:

[0336] Server: Determines the reward for the informant's account and reflects it in cash or points.

[0337] In this way, through specific processing steps, the user can easily obtain the information they need in a form optimized for their emotions, and the information provider can easily obtain profits from the system.

[0338] Example 2

[0339] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Current information provision systems can generate useful information based on images and location information provided by users, but this information is often provided without considering the user's emotional state. This makes it difficult to provide optimal information tailored to the user's needs and circumstances. Furthermore, the selling price of the generated information is not optimized, and rewards to information providers are not efficiently provided. These issues need to be resolved.

[0341] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for assigning location information to a captured image, a means for uploading the image and location information to the server, a means for using a generation AI in the server to generate useful information based on the image and location information, a means for setting a selling price for the generated information, a means for saving the generated information in a database and making it searchable, a means for searching for information from the database and displaying the results, a means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and a means for using an emotion engine that recognizes emotions from the image and location information uploaded by the user. This makes it possible to provide optimal information according to the user's emotional state and efficiently set the selling price of the information. Furthermore, it also becomes possible to efficiently reward information providers.

[0342] "Photographed images" are photos or videos taken by a user using a mobile device such as a smartphone or tablet.

[0343] "Location information" refers to latitude and longitude data obtained using the GPS function of a mobile device.

[0344] A "server" is a computer system that has the ability to receive, process, store, retrieve, and display data.

[0345] "Generative AI" is an artificial intelligence model that uses image analysis algorithms to automatically generate useful information from images and location information.

[0346] "Useful information" is information that is valuable to users, such as road congestion, congestion at shopping facilities and tourist spots, parking availability, real-time weather information, and the blooming status of plants.

[0347] The "sale price" is the price at which the generated useful information is offered to the user.

[0348] A "database" is an information management system that stores generated information and keeps it searchable.

[0349] The "emotion engine" is an analytical engine that recognizes a user's emotions from images and location information.

[0350] An "information provider" is a user who contributes to the generation of useful information by providing captured images and location information.

[0351] "Upload" means sending image and location information data from the user's mobile terminal to the server.

[0352] This invention is a system that automatically generates useful information using a generation AI based on images and location information taken by users, sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for carrying out the invention are described below.

[0353] User capture and upload of images and location information

[0354] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[0355] Server-based data processing and information generation

[0356] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[0357] Road congestion

[0358] Congestion levels at sales facilities and tourist spots

[0359] Parking availability

[0360] Real-time weather information

[0361] Flowering status of plants

[0362] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[0363] Recognizing user emotions with an emotion engine

[0364] The server is also equipped with an emotion engine. The emotion engine uses technology to recognize a user's emotions by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data included in images uploaded by the user to identify the user's emotions (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation of the generated useful information.

[0365] Searching for and purchasing information

[0366] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[0367] Purchase and Use of Information

[0368] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[0369] Specific examples

[0370] Example 1: Combining tourist destination congestion and user sentiment

[0371] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[0372] 2. Server: Receives images and location information and passes them to the generation AI.

[0373] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[0374] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[0375] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[0376] Example 2: Combining parking availability and user sentiment

[0377] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[0378] 2. Server: Receives images and location information and passes them to the generation AI.

[0379] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[0380] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[0381] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[0382] Prompt Sentence Examples

[0383] "Please tell me about the congestion situation at tourist spot A. Please also take user emotions into consideration when displaying this information."

[0384] "Provide optimal information based on parking availability and user sentiment."

[0385] This system provides an efficient means for widely sharing valuable information in real time, and can improve the user experience by providing the most appropriate information according to the user's emotions.

[0386] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0387] Step 1:

[0388] A user uses a smartphone or tablet to take an image (photo or video) at a specific location. When taking a photo, the device automatically uses its GPS function to obtain location information (latitude and longitude). The captured image (input) and the obtained location information (input) are uploaded to a server (output) via a dedicated application. Specifically, the user presses the "take a photo" button in the application, and then presses the "upload" button after taking the photo.

[0389] Step 2:

[0390] The server receives the image and location information uploaded by the user (input). The server temporarily stores the received data and verifies that the image and location information were received correctly (output). Specifically, the server receives the HTTP request and analyzes the image file and location information.

[0391] Step 3:

[0392] The server sends the received image and location information to the generating AI (input). This transmission is done by calling the generating AI's API (output). Specifically, the server executes an HTTP POST request to send data to the generating AI model's endpoint.

[0393] Step 4:

[0394] The generating AI analyzes the received images and generates useful information based on the location information (input). The generated useful information (output) includes road congestion, the congestion level of sales facilities, parking availability, real-time weather information, and the blooming status of plants. Specifically, the generating AI applies image analysis algorithms to generate various types of information.

[0395] Step 5:

[0396] After receiving the information returned by the generation AI, the server executes the emotion engine (input). The emotion engine analyzes the facial expressions and voice data in the image to identify the user's emotional state (output). Specifically, the emotion engine recognizes the facial expressions in the image and determines the user's emotions, such as whether they are happy or sad.

[0397] Step 6:

[0398] The server stores the generated information and the user's emotional state in a database (input) and keeps it searchable (output). At the same time, it uses a pricing algorithm to set a selling price for the generated information. Specifically, the server sets a price based on the type and demand of the generated information and stores the information in the database.

[0399] Step 7:

[0400] When a user specifies a specific location using the application's map interface (input), the server searches the database for registered information about the area around that location and displays it to the user (output). Based on the emotion engine, the server prioritizes and displays information that is highly relevant to the user's current emotional state. Specifically, upon receiving the user's specification, the server retrieves information from the database and displays it in the application.

[0401] Step 8:

[0402] The user selects the information they need from the displayed search results and completes the purchase process (input). Once the purchase is complete, the server provides the information to the user and awards a reward to the information provider (output). Specifically, the user selects information from the search results and presses the "Purchase" button to complete the payment. The server then sends the purchase information to the user and updates the information provider's account.

[0403] Step 9:

[0404] Once the purchase procedure is completed, the server grants a reward to the information provider (input). This reward is reflected in the information provider's account and can be received as cash or points (output). Specifically, the server updates the information provider's account and adds the amount of the reward or points.

[0405] (Application example 2)

[0406] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0407] Previous systems were unable to take the user's emotions into account when generating useful information from images and location information taken by the user. This resulted in the information provided not necessarily being optimal for the user's current situation and emotions, potentially resulting in a poor user experience. Furthermore, when providing information related to a specific field, optimal recommendations based on the user's emotions and situation were lacking.

[0408] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0409] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for saving the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for analyzing emotions to provide search results related to a specific field, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for providing information related to the field in which the order procedure is included. This makes it possible to provide information optimized for the user's emotions and current situation, thereby improving the user experience.

[0410] An "image" is a photograph or video taken by a user using a mobile device.

[0411] "Location information" is latitude and longitude data used to identify the location where a captured image was taken.

[0412] The "server" is a central processing unit that receives images and location information uploaded by users, analyzes them, and generates information.

[0413] "Useful information" is information that is valuable to the user, and specifically includes information on congestion, availability, weather, and the like.

[0414] The "means for setting a selling price" is an algorithm or system for setting a price for the generated useful information and selling it.

[0415] A "database" is a system for storing useful information that has been generated and managing it so that it can be searched later.

[0416] "Emotion analysis means" is a technology for analyzing the user's emotions and improving the appropriateness of the information provided.

[0417] "Order procedure" refers to a series of procedures that a user must follow to purchase the provided information and complete the transaction.

[0418] An "information provider" is a user who provides useful information by uploading images and location information, and is eligible to receive a reward.

[0419] "Generative AI" is an artificial intelligence technology that analyzes images and location information provided by users and generates useful information.

[0420] System Overview

[0421] This invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and provides that information. In this system, users can easily upload information, and useful information is generated in real time based on that information. It also incorporates emotion analysis means for analyzing user emotions, providing more optimized information. Below, we will explain specific embodiments of this system.

[0422] Required Hardware and Software

[0423] Mobile devices (smartphones, tablets): Used by users to take images and obtain location information.

[0424] GPS function: Used to automatically obtain location information.

[0425] Generative AI models (e.g., GPT-4, DALL-E, etc.): Analyze images and location information and use them to generate useful information.

[0426] Sentiment analysis engine (e.g., Affectiva): Used to analyze user emotions and optimize the information provided.

[0427] A database management system (e.g., MySQL) is used to store and search the useful information generated.

[0428] Cloud server (e.g., AWS): Used to receive data from users and run generative AI models and sentiment analysis engines.

[0429] Detailed System Operation

[0430] The user operates the device to take pictures at a specific location, and the location information is automatically acquired using the GPS function. This data is then uploaded to a cloud server via a dedicated application.

[0431] The cloud server passes the received images and location information to the generative AI model, which then performs image analysis. As a result of the analysis, useful information is generated based on that data. The generated information is then passed to an emotion analysis engine, which analyzes the user's emotions from the uploaded images. Based on the analysis results, the information provided is optimized to the user's current emotions and situation.

[0432] The generated information is stored in a database and made searchable. When a user searches for information, the most relevant information is displayed preferentially based on the results of sentiment analysis. This information is provided when the user completes a purchase, and the information provider is rewarded.

[0433] Specific examples

[0434] Example 1: Combining restaurant congestion and user sentiment

[0435] 1. User: Takes a photo at a restaurant and uploads it to the server through the app.

[0436] 2. Server: Receives images and location information and passes them to the generation AI.

[0437] 3. Generative AI: Analyzes the restaurant's occupancy status and returns the information to the server.

[0438] 4. Emotion Engine: Analyzes user emotions from uploaded images and recommends restaurants with fast service if, for example, you are tired.

[0439] 5. Server: Sets the selling price of the information and stores it in the database. When a user searches, restaurant information that is best suited to the user's tired situation is displayed preferentially.

[0440] Prompt Sentence Examples

[0441] "Please tell me the current congestion situation around Tokyo Tower and recommended food delivery spots."

[0442] This embodiment enables the provision of information optimized according to the user's emotions and circumstances, improving the user experience. Information providers can also easily obtain benefits, and a means is provided for effectively and widely sharing valuable information in real time.

[0443] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0444] Step 1:

[0445] A user takes an image at a specific location using a mobile device. After taking the image, the location information (latitude and longitude) is automatically acquired using the device's GPS function.

[0446] Input: Photo or video, location

[0447] Output: Image data, location data

[0448] Step 2:

[0449] The device uploads the captured images and location information to a cloud server via a dedicated application, making this operation extremely convenient for users.

[0450] Input: Image data, location data

[0451] Output: Data sent to the server

[0452] Step 3:

[0453] The server receives images and location information uploaded by users and passes them to a generative AI model, which uses image analysis algorithms to generate useful information.

[0454] Input: Image data, location data

[0455] Output: Generated useful information (e.g., crowd situation, popular meals)

[0456] Step 4:

[0457] The emotion analysis engine analyzes the user's emotions from the uploaded images, for example, identifying emotions such as joy or fatigue from facial expression analysis and text content.

[0458] Input: Image data

[0459] Output: User emotion data

[0460] Step 5:

[0461] The server receives the generated useful information and the sentiment analysis results, stores the information in a database, and sets a selling price for the information based on a pricing algorithm.

[0462] Input: Generated useful information, user emotion data

[0463] Output: Saved data, set selling price

[0464] Step 6:

[0465] When a user specifies a location using the application's map interface, the server searches the database for registered information near the specified location and displays the results, prioritizing relevant information based on the results of sentiment analysis.

[0466] Input: Search query (location specified), user emotion data

[0467] Output: Search results (information list)

[0468] Step 7:

[0469] The user selects the information they need from the search results and completes the purchase process. The server completes the purchase process and rewards the information provider. The reward is reflected in the information provider's account and can be received as cash or points.

[0470] Input: Purchase procedure information, user account information

[0471] Output: Purchase completion notification, information provider's reward data

[0472] This series of processes allows users to easily obtain useful information based on their own captured images and location information. Furthermore, by combining it with sentiment analysis, it becomes possible to provide information optimized for the user's current situation, significantly improving the user experience. Information providers can also easily benefit from this system, providing a means to effectively share valuable information in real time.

[0473] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0474] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0475] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0476] [Second embodiment]

[0477] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0478] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0479] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0480] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0481] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0482] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0483] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0484] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0485] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0486] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0487] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0488] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0489] The present invention is a system that automatically generates useful information using AI based on images and location information taken by users, and sells that information. Specific embodiments for implementing the present invention are described below.

[0490] User capture and upload of images and location information

[0491] Users use mobile devices such as smartphones and tablets to take photos (or videos) at specific locations. For example, users may take photos at tourist spots, shopping malls, parking lots, etc. When this happens, the device automatically uses its GPS function to obtain location information (latitude and longitude).

[0492] After taking a photo, the user uploads the photo and location information to the server using a dedicated application. The user's operation is extremely simple, and the data can be sent simply by following the application's instructions.

[0493] Server-based data processing and information generation

[0494] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[0495] Road congestion

[0496] Congestion levels at stores and tourist spots

[0497] Parking availability

[0498] Real-time weather information

[0499] Flowering status

[0500] For example, the generative AI can analyze the number of cars and the density of people in an image to determine the congestion level of roads and tourist spots. It can also determine the availability of parking spaces based on the usage status of parking spaces visible in the image. It can also obtain weather information that can be determined from the image.

[0501] The information generated by the AI ​​is returned to the server, which stores it in a database. At the same time, the system also includes a function to automatically set a selling price based on the scarcity and demand of the information.

[0502] Searching for and purchasing information

[0503] When a user wants to search for specific information, they specify the location they want to know about using the application's map interface. For example, if a user wants to know the crowd situation at a particular tourist spot, they specify that point on the map. The server then searches the database for information registered near the specified point and displays the results to the user.

[0504] The user selects the information they need from the displayed results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[0505] Specific examples

[0506] Example 1: Providing information on the congestion status of tourist spots

[0507] 1. User: Takes and uploads a photo at tourist spot A.

[0508] 2. Server: Receives images and location information and passes them to the generation AI.

[0509] 3. Generative AI: Analyzes the level of congestion at tourist spot A and returns the information to the server.

[0510] 4. Server: Sets the selling price of the information and stores it in the database.

[0511] If another user wants to know the congestion situation at tourist spot A,

[0512] 1. User: Specify tourist spot A via the map interface.

[0513] 2. Server: Search and display information about tourist spot A.

[0514] 3. User: Purchases information and obtains congestion status provided by the server.

[0515] Example 2: Providing parking availability information

[0516] 1. User: Takes and uploads a photo in the parking lot.

[0517] 2. Server: Receives images and location information and passes them to the generation AI.

[0518] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[0519] 4. Server: Sets the selling price of the information and stores it in the database.

[0520] If another user wants to know the availability of that parking space,

[0521] 1. User: Specify parking via map interface.

[0522] 2. Server: Searches and displays information about nearby parking lots.

[0523] 3. User: Purchases information and obtains availability provided by the server.

[0524] These embodiments allow users to easily obtain the information they need and provide benefits to information providers. The system provides an efficient means for widely sharing valuable information in real time.

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] User: Uses a smartphone or tablet to take images (photos or videos) at specific locations, such as tourist spots or parking lots.

[0528] Step 2:

[0529] Device: When an image is taken, the location information (latitude and longitude) is automatically obtained using the GPS function.

[0530] Step 3:

[0531] User: Uploads the captured images and their location information to the server via a dedicated application.

[0532] Step 4:

[0533] Server: Receives images and location information uploaded by users.

[0534] Step 5:

[0535] Server: Passes the received data to the generation AI.

[0536] Step 6:

[0537] Generative AI: Using image analysis algorithms to analyze elements within an image and generate useful information such as:

[0538] Identify road congestion based on the number of cars and people.

[0539] Analyzes the density of people to determine how crowded stores and tourist spots are.

[0540] To identify parking availability based on parking space usage.

[0541] Determine the weather (sunny, cloudy, rainy, etc.) from the photo.

[0542] Analyze the type of flower and its flowering status.

[0543] Step 7:

[0544] Generative AI: Returns the generated useful information to the server.

[0545] Step 8:

[0546] Server: Sets a selling price based on the generated information, taking into account factors such as scarcity and demand for the information.

[0547] Step 9:

[0548] Server: Stores the generated information in a database.

[0549] Step 10:

[0550] Server: Makes the stored information searchable.

[0551] Step 11:

[0552] User (information seeker): Uses the application's map interface to search for information by specifying the location of interest.

[0553] Step 12:

[0554] Server: Searches for information stored in a database near a specified location.

[0555] Step 13:

[0556] Server: Displays the information to the user as search results.

[0557] Step 14:

[0558] User: Select the necessary information from the search results and proceed with the purchase.

[0559] Step 15:

[0560] Server: Provides the user with the relevant information once the purchase is complete.

[0561] Step 16:

[0562] Server: Determines the reward for the informant and reflects it in the account.

[0563] In this way, the system allows users to easily obtain the information they need through specific processing steps, and also allows information providers to easily obtain profits.

[0564] Example 1

[0565] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0566] In modern society, there is a growing demand for useful information obtained in real time. However, existing systems have limitations in the means to efficiently collect, analyze, and provide such information. In particular, systems that combine images and location information to provide information intuitively and immediately have not yet been fully developed. Therefore, an objective of the present invention is to provide a system that automatically generates useful information in real time using images and location information taken by users and makes that information available for sale.

[0567] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0568] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to a data storage, means for generating useful information in the data storage based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a search system and making it searchable, means for searching for information from the search system and displaying the results, and means for purchasing the searched information and providing a reward to the information provider when the purchase procedure is completed. This allows users to easily obtain useful information in real time, and enables information providers to easily earn profits.

[0569] "Means for adding location information to captured images" refers to a function that uses the camera and GPS functions built into the user's mobile device to automatically add location information such as latitude and longitude to images when they are taken.

[0570] The "means for uploading the images and location information to data storage" refers to a function that allows a user to send the captured images and their location information to a server via a dedicated application and store them in cloud-based data storage.

[0571] The "means for generating useful information based on the images and location information in the data storage" refers to a function that uses a generation AI model in the server to analyze the received images and location information and generate useful information (such as congestion status and parking availability).

[0572] The "means for setting the selling price of the generated information" is an algorithm for evaluating the value of the information generated by the server and automatically calculating an appropriate selling price based on the rarity and demand of the information.

[0573] "Means for storing the generated information in a search system and making it searchable" refers to a function that stores the generated useful information in a database so that users can easily find it through a search system.

[0574] "Means for searching for information from the search system and displaying the results" refers to a function in which, when a user enters specific information into the search system, the server searches for the corresponding information from the database and displays the results on the user's terminal.

[0575] "Means for purchasing searched information and rewarding the information provider once the purchase procedure is completed" refers to a system that manages the payment process for purchasing information found by a user and pays a reward to the user who provided the information once the purchase is completed.

[0576] "Traffic congestion status" refers to information about the density of vehicles and the degree of congestion on specific roads, intersections, etc.

[0577] "Crowding levels at hubs and tourist destinations" refers to information about the density of people and the degree of congestion at specific facilities or areas.

[0578] "Parking area availability" is information about the number and location of available parking spaces in a particular parking lot.

[0579] "Real-time weather information" refers to information about current weather conditions, such as temperature and precipitation, in a particular area.

[0580] "Flowering status of plants" is information about the progress of flowering and the best time to see a particular plant.

[0581] A "generative AI model" is an artificial intelligence algorithm with functions such as image analysis and data generation, and in particular uses computer vision and natural language processing to generate useful information.

[0582] This invention is a system that automatically generates useful information using a generative AI model based on images taken by users and location information, and sells that information. Specific embodiments for implementing the invention are described below.

[0583] Overall system configuration

[0584] Users take photos and videos at specific locations using mobile devices such as smartphones and tablets. At this time, the device uses its GPS function to obtain location information (latitude and longitude). After taking the photos, users upload the images and location information to a server using a dedicated application.

[0585] The server receives images and location information uploaded by users and automatically generates useful information based on that information using a generative AI model. The generated information is stored in a database and made searchable, with a selling price automatically set based on the information's rarity and demand. Finally, when the user searches for and purchases the information and the purchase process is completed, the information provider is rewarded.

[0586] Hardware and software used

[0587] Hardware

[0588] Smartphones and tablets (e.g. iPhone, Android smartphone, iPad)

[0589] Cloud servers (e.g. AWS, Google Cloud, Microsoft Azure)

[0590] software

[0591] Camera app and GPS function on mobile devices

[0592] Dedicated app (iOS app, Android app)

[0593] Generative AI models (e.g., OpenAI GPT-4)

[0594] Database management systems (e.g., MySQL, PostgreSQL)

[0595] Specific details of data processing and calculation

[0596] Images taken by users on their mobile devices and the acquired location information are uploaded to the server via a dedicated app. The server then performs the following processes.

[0597] 1. Data integrity check

[0598] The server checks the consistency of the received image data and location information, checking for errors and omissions.

[0599] 2. Data Preprocessing

[0600] The server converts the image data into a format that can be passed to the generative AI model (e.g., JSON format).

[0601] 3. Analysis and information generation using generative AI models

[0602] The generative AI model analyzes features in an image (e.g., number of cars, human density, weather, etc.) and generates useful information (e.g., congestion level, parking availability, etc.).

[0603] Example of a prompt to be passed to the AI ​​generator:

[0604] "Please analyze the congestion situation at store X from this image."

[0605] "Please identify the availability of this parking space from this image."

[0606] 4. Information Storage and Pricing

[0607] The server stores the generated information in a database and makes it searchable.

[0608] The server automatically sets the selling price based on the scarcity and demand of the information.

[0609] Specific examples

[0610] Example 1: Providing information on the congestion status of tourist spots

[0611] 1. The user takes a photo at tourist spot A and uploads it to the server via a dedicated app.

[0612] 2. The server receives the image and location information and passes it to the generative AI model.

[0613] 3. The generative AI model analyzes the level of congestion at tourist destination A and returns the results to the server.

[0614] 4. The server sets the selling price of the information and stores it in the database.

[0615] If another user wants to know the congestion situation at tourist spot A,

[0616] 1. The user specifies tourist spot A via the map interface.

[0617] 2. The server searches for and displays information about tourist spot A.

[0618] 3. The user purchases information and obtains congestion status provided by the server.

[0619] Example 2: Providing parking availability information

[0620] 1. The user takes a photo in the parking lot and uploads it to the server via a dedicated app.

[0621] 2. The server receives the image and location information and passes it to the generative AI model.

[0622] 3. The generative AI model analyzes the parking availability and returns the results to the server.

[0623] 4. The server sets the selling price of the information and stores it in the database.

[0624] If another user wants to know the availability of that parking space,

[0625] 1. The user specifies a parking spot via the map interface.

[0626] 2. The server searches for and displays information about nearby parking lots.

[0627] 3. The user purchases the information and obtains the availability provided by the server.

[0628] As described above, this system allows users to easily obtain useful information in real time, and information providers can easily benefit from it. This system provides an efficient means for widely sharing valuable information.

[0629] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0630] Step 1:

[0631] User capture and upload of images and location information

[0632] Operation overview

[0633] Input: Images taken by the user on a mobile device and acquired location information

[0634] How it works: The user launches the dedicated application and takes a photo or video at a specific location. At this time, the location information (latitude and longitude) is automatically obtained using the mobile device's GPS function. After taking the photo, the user presses the send button in the dedicated application to upload the image and location information to the server.

[0635] Output: Image data and location information sent to the server

[0636] Step 2:

[0637] Data reception and preprocessing by the server

[0638] Operation overview

[0639] Input: Image data and location information uploaded by the user

[0640] How it works: The server receives an HTTP request and retrieves image and location data. The server checks the data for integrity and errors or omissions. It then converts the image data into an appropriate format (e.g., JSON) to be passed to the generative AI model.

[0641] Output: Preprocessed image data and location information

[0642] Step 3:

[0643] Information generation by generative AI

[0644] Operation overview

[0645] Input: Preprocessed image data and location information

[0646] How it works: The server sends the preprocessed data to a generative AI model. The generative AI model analyzes features in the image (e.g., number of cars, human density, weather, etc.) and automatically generates useful information (e.g., congestion level, parking availability, etc.). Examples of prompts input to the generative AI model include "Please analyze the congestion level at Store X from this image" and "Please identify the availability of this parking lot from this image."

[0647] Output: Useful information as analysis results (such as congestion status and parking availability)

[0648] Step 4:

[0649] Server-based information storage and pricing

[0650] Operation overview

[0651] Input: Useful information from the analysis results obtained from the generative AI model

[0652] How it works: The server stores the generated information in a database. It then applies an algorithm to automatically set the selling price based on the scarcity and demand of the information, and determines an appropriate selling price. The information, along with the selling price, is made searchable.

[0653] Output: Useful information stored in the database and its selling price

[0654] Step 5:

[0655] User search and purchase

[0656] Operation overview

[0657] Input: Search criteria specified by the user (e.g., the congestion status of a specific location)

[0658] How it works: The user specifies the location they want to know about using the map interface in the dedicated app. The server searches the database for information close to the specified location and displays the results to the user. The user selects the information they need from the displayed information and completes the purchase procedure. Once payment is complete, the server provides the information to the user and simultaneously rewards the information provider.

[0659] Output: Useful information provided to the user and the reward given.

[0660] (Application example 1)

[0661] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0662] In modern urban environments, safety management of public areas and private property is a critical issue. In particular, during large-scale events and disasters, it is necessary to grasp safety situations in real time and respond quickly, but current systems are insufficient in this regard. Furthermore, there is no established method for efficiently collecting this safety information and appropriately rewarding data providers. Furthermore, there is a lack of ways for ordinary users to contribute to public safety using their own mobile devices.

[0663] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0664] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for evaluating the safety of a specific area based on the generated information and issuing an alert. This allows users to easily collect and provide safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[0665] output

[0666] The "means for assigning location information to a captured image" refers to a device or software that provides a function for automatically associating location information such as latitude and longitude with an image captured by a user.

[0667] The "means for uploading the images and location information to the server" refers to a device or software that includes a network connection and communication protocol for transmitting images taken by a user and their associated location information to the server.

[0668] The "means for generating useful information based on the images and location information at the server" refers to an algorithm or computer program that analyzes the received images and location information and generates important data or insights based on specific conditions.

[0669] The "means for setting the selling price of the generated information" is an algorithm or computer program that determines an appropriate selling price based on the market value and demand for the generated useful information.

[0670] "Means for storing the generated information in a database and making it searchable" refers to a database management system that efficiently stores the generated information and makes it easily searchable and accessible later.

[0671] The "means for searching information from the database and displaying the results" refers to an interface and algorithms for extracting relevant information from the database based on a search query from a user and displaying the results to the user.

[0672] "Means for purchasing searched information and rewarding the information provider when the purchase procedure is completed" is a system that automatically awards money or points to the user who provided the information when the transaction is completed after the user purchases the information.

[0673] The "means for assessing the safety of a specific area based on the generated information and issuing an alert" refers to an algorithm and communication device for analyzing the generated information, assessing the safety of a specific area, and issuing an alert to relevant parties as necessary.

[0674] output

[0675] This invention is a system that uses AI to automatically generate useful information based on images and location information taken by users, and provides and sells that information. This system is particularly designed for application in the field of safety management, and is capable of issuing area safety assessments and warnings in real time.

[0676] 1. Acquisition and upload of images and location information by user devices

[0677] Users use mobile devices such as smartphones and tablets to take photos at specific locations. For example, users may take photos in public areas or on private property. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the photo, users upload the photo and location information to a server using a dedicated application. This allows users to easily provide the necessary data to the system.

[0678] 2. Data processing and information generation by the server

[0679] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[0680] Safety assessment of specific areas (detection of suspicious objects, determination of dangerous areas, etc.)

[0681] Real-time information related to public safety (such as congestion and traffic conditions)

[0682] For example, the generating AI analyzes whether an image contains any suspicious objects and uses that information to evaluate the safety of the area. It also evaluates the safety of the surrounding environment in real time based on data obtained from location information. The generated information is returned to the server, which stores it in a database. At the same time, it automatically sets a selling price based on the scarcity and demand of the information.

[0683] 3. Searching for and Purchasing Information

[0684] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. For example, if a user wants to know the safety status of a particular area, they specify that location on the map. The server searches the database for information registered near the specified location and displays the results to the user. The user selects the information they need from the displayed results and proceeds with the purchase. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[0685] Hardware and software used

[0686] Hardware: Smartphones, tablets

[0687] Software: Dedicated application, server, generative AI, Google Cloud Vision API, Google Maps API, requests library, database management system

[0688] Specific examples

[0689] Example 1: Public Area Safety Assessment

[0690] 1. User: Finds a suspicious object in the park, takes a photo, and uploads it. The location information is automatically captured.

[0691] 2. Server: Receives images and location information and passes them to the generation AI.

[0692] 3. Generative AI: Performs a safety assessment of the park and determines that it is a dangerous area.

[0693] 4. Server: Stores the safety assessment results in a database and issues alerts as necessary.

[0694] Prompt Sentence Examples

[0695] "Generate the following information based on the photo and GPS location:

[0696] Is there anything suspicious in the image?

[0697] Assessment of local crime prevention and safety conditions

[0698] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0699] output

[0700] Step 1:

[0701] A user takes a photo at a specific location using a device such as a smartphone or tablet. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). The input is the captured image and its location information. As a result, the image taken by the user is saved along with the location information.

[0702] Step 2:

[0703] After taking a photo, the user uses a dedicated application to upload the photo and location information to the server. The input is the image and location information acquired in step 1. The output is the image and location information stored on the server. The server receives this information and prepares it for the next processing step.

[0704] Step 3:

[0705] The server passes the received image and location information to the generation AI. The input is the image and location information stored on the server. The generation AI uses image analysis algorithms to detect suspicious objects in the uploaded image and evaluate the safety of the area. This analysis is performed by detecting specific objects in the image and matching them with location information. The output is the results of the image analysis (e.g., whether or not there is a suspicious object, the safety evaluation of the area, etc.).

[0706] Step 4:

[0707] The server sets the selling price of the generated information based on the analysis results received from the generation AI. The input is the image analysis results. This automatically determines an appropriate selling price based on market value and demand. The output is the set selling price.

[0708] Step 5:

[0709] The server stores the generated information and its selling price in a database, making it searchable. The input is the generated information and the set selling price. This allows the database to be searchable and accessible by users later, and the output is the information stored in the database.

[0710] Step 6:

[0711] When a user searches for specific information, they use the application's map interface to specify the location they want to know about. The input is the user's search query (e.g., safety status in a particular area). The server searches a database for relevant information based on the query and displays the results. The output is the displayed search results.

[0712] Step 7:

[0713] The user selects the information they need from the displayed search results and completes the purchase process. The input is the selected information and the purchase process information. Once the purchase process is completed, the server provides the information to the user and awards a reward to the information provider. This allows the provider to receive a reward for providing the information, and the user can obtain the information they need. The output is the information provided to the user and the information provider with the award.

[0714] The above processing steps enable users to collect and provide useful safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[0715] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0716] The present invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for implementing the present invention will be described below.

[0717] User capture and upload of images and location information

[0718] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[0719] Server-based data processing and information generation

[0720] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[0721] Road congestion

[0722] Congestion levels at stores and tourist spots

[0723] Parking availability

[0724] Real-time weather information

[0725] Flowering status

[0726] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[0727] Recognizing user emotions with an emotion engine

[0728] The present invention further incorporates an emotion engine. The emotion engine uses technology to recognize a user's emotion by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data contained in images uploaded by the user to identify the user's emotion (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation method of the generated useful information.

[0729] Searching for and purchasing information

[0730] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[0731] Purchase and Use of Information

[0732] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[0733] Specific examples

[0734] Example 1: Combining tourist destination congestion and user sentiment

[0735] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[0736] 2. Server: Receives images and location information and passes them to the generation AI.

[0737] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[0738] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[0739] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[0740] Example 2: Combining parking availability and user sentiment

[0741] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[0742] 2. Server: Receives images and location information and passes them to the generation AI.

[0743] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[0744] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[0745] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[0746] This allows users to easily obtain the information they need in a form optimized for their emotions, and information providers can easily earn profits. This system provides an efficient means for widely sharing valuable information in real time.

[0747] The processing flow will be explained below.

[0748] Step 1:

[0749] User: Take an image (photo or video) with a smartphone or tablet at a specific location (e.g., a tourist spot or parking lot).

[0750] Step 2:

[0751] Device: The location information (latitude and longitude) of the captured image is automatically obtained using the GPS function.

[0752] Step 3:

[0753] User: The captured images and location information are uploaded to the server through a dedicated application. The application has a user-friendly interface and is easy to operate.

[0754] Step 4:

[0755] Server: Receives images and location information uploaded by users and temporarily stores them in a database.

[0756] Step 5:

[0757] Server: Passes the received image and location information to the generation AI, which runs an image analysis algorithm to analyze elements within the image.

[0758] Step 6:

[0759] Generative AI: Analyzes elements in an image and generates useful information. Examples include:

[0760] Analyze the number of cars and people to identify road congestion.

[0761] Analyzes the density of people to determine how crowded stores and tourist spots are.

[0762] Analyze parking space usage to identify available parking spaces.

[0763] Determine the weather (sunny, cloudy, rainy, etc.) from the image.

[0764] Analyze the type of flower and its flowering status.

[0765] Step 7:

[0766] Generative AI: Returns the generated useful information to the server.

[0767] Step 8:

[0768] Server: Sets a selling price based on the useful information received, taking into account factors such as scarcity and demand for the information.

[0769] Step 9:

[0770] Server: Passes the generated information to the emotion engine and provides the information necessary for analyzing user emotions.

[0771] Step 10:

[0772] Emotion engine: Analyzes image and text content to recognize the user's emotional state (e.g., joy, sadness, surprise, etc.).

[0773] Step 11:

[0774] Emotion engine: Optimizes the presentation of generated information based on the user's emotional state, for example, suggesting less congested routes for users who are feeling stressed.

[0775] Step 12:

[0776] Server: Based on the results of the emotion engine, the optimized information is stored in a database and made searchable.

[0777] Step 13:

[0778] User (information searcher): Uses the map interface of the application to search for information by specifying the location they want to know about. For example, if a user wants to know the congestion status of a tourist spot, they specify that location on the map.

[0779] Step 14:

[0780] Server: Searches the database for registered information near the specified location and displays the results to the user.

[0781] Step 15:

[0782] User: Selects the information they need from the search results and completes the purchase. They also check the information presented based on their emotional state.

[0783] Step 16:

[0784] Server: Once the purchase process is complete, the server provides the relevant information to the user and simultaneously initiates the reward process for the information provider.

[0785] Step 17:

[0786] Server: Determines the reward for the informant's account and reflects it in cash or points.

[0787] In this way, through specific processing steps, the user can easily obtain the information they need in a form optimized for their emotions, and the information provider can easily obtain profits from the system.

[0788] Example 2

[0789] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0790] Current information provision systems can generate useful information based on images and location information provided by users, but this information is often provided without considering the user's emotional state. This makes it difficult to provide optimal information tailored to the user's needs and circumstances. Furthermore, the selling price of the generated information is not optimized, and rewards to information providers are not efficiently provided. These issues need to be resolved.

[0791] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for assigning location information to a captured image, a means for uploading the image and location information to the server, a means for using a generation AI in the server to generate useful information based on the image and location information, a means for setting a selling price for the generated information, a means for saving the generated information in a database and making it searchable, a means for searching for information from the database and displaying the results, a means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and a means for using an emotion engine that recognizes emotions from the image and location information uploaded by the user. This makes it possible to provide optimal information according to the user's emotional state and efficiently set the selling price of the information. Furthermore, it also becomes possible to efficiently reward information providers.

[0792] "Photographed images" are photos or videos taken by a user using a mobile device such as a smartphone or tablet.

[0793] "Location information" refers to latitude and longitude data obtained using the GPS function of a mobile device.

[0794] A "server" is a computer system that has the ability to receive, process, store, retrieve, and display data.

[0795] "Generative AI" is an artificial intelligence model that uses image analysis algorithms to automatically generate useful information from images and location information.

[0796] "Useful information" is information that is valuable to users, such as road congestion, congestion at shopping facilities and tourist spots, parking availability, real-time weather information, and the blooming status of plants.

[0797] The "sale price" is the price at which the generated useful information is offered to the user.

[0798] A "database" is an information management system that stores generated information and keeps it searchable.

[0799] The "emotion engine" is an analytical engine that recognizes a user's emotions from images and location information.

[0800] An "information provider" is a user who contributes to the generation of useful information by providing captured images and location information.

[0801] "Upload" means sending image and location information data from the user's mobile terminal to the server.

[0802] This invention is a system that automatically generates useful information using a generation AI based on images and location information taken by users, sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for carrying out the invention are described below.

[0803] User capture and upload of images and location information

[0804] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[0805] Server-based data processing and information generation

[0806] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[0807] Road congestion

[0808] Congestion levels at sales facilities and tourist spots

[0809] Parking availability

[0810] Real-time weather information

[0811] Flowering status of plants

[0812] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[0813] Recognizing user emotions with an emotion engine

[0814] The server is also equipped with an emotion engine. The emotion engine uses technology to recognize a user's emotions by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data included in images uploaded by the user to identify the user's emotions (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation of the generated useful information.

[0815] Searching for and purchasing information

[0816] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[0817] Purchase and Use of Information

[0818] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[0819] Specific examples

[0820] Example 1: Combining tourist destination congestion and user sentiment

[0821] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[0822] 2. Server: Receives images and location information and passes them to the generation AI.

[0823] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[0824] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[0825] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[0826] Example 2: Combining parking availability and user sentiment

[0827] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[0828] 2. Server: Receives images and location information and passes them to the generation AI.

[0829] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[0830] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[0831] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[0832] Prompt Sentence Examples

[0833] "Please tell me about the congestion situation at tourist spot A. Please also take user emotions into consideration when displaying this information."

[0834] "Provide optimal information based on parking availability and user sentiment."

[0835] This system provides an efficient means for widely sharing valuable information in real time, and can improve the user experience by providing the most appropriate information according to the user's emotions.

[0836] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0837] Step 1:

[0838] A user uses a smartphone or tablet to take an image (photo or video) at a specific location. When taking a photo, the device automatically uses its GPS function to obtain location information (latitude and longitude). The captured image (input) and the obtained location information (input) are uploaded to a server (output) via a dedicated application. Specifically, the user presses the "take a photo" button in the application, and then presses the "upload" button after taking the photo.

[0839] Step 2:

[0840] The server receives the image and location information uploaded by the user (input). The server temporarily stores the received data and verifies that the image and location information were received correctly (output). Specifically, the server receives the HTTP request and analyzes the image file and location information.

[0841] Step 3:

[0842] The server sends the received image and location information to the generating AI (input). This transmission is done by calling the generating AI's API (output). Specifically, the server executes an HTTP POST request to send data to the generating AI model's endpoint.

[0843] Step 4:

[0844] The generating AI analyzes the received images and generates useful information based on the location information (input). The generated useful information (output) includes road congestion, the congestion level of sales facilities, parking availability, real-time weather information, and the blooming status of plants. Specifically, the generating AI applies image analysis algorithms to generate various types of information.

[0845] Step 5:

[0846] After receiving the information returned by the generation AI, the server executes the emotion engine (input). The emotion engine analyzes the facial expressions and voice data in the image to identify the user's emotional state (output). Specifically, the emotion engine recognizes the facial expressions in the image and determines the user's emotions, such as whether they are happy or sad.

[0847] Step 6:

[0848] The server stores the generated information and the user's emotional state in a database (input) and keeps it searchable (output). At the same time, it uses a pricing algorithm to set a selling price for the generated information. Specifically, the server sets a price based on the type and demand of the generated information and stores the information in the database.

[0849] Step 7:

[0850] When a user specifies a specific location using the application's map interface (input), the server searches the database for registered information about the area around that location and displays it to the user (output). Based on the emotion engine, the server prioritizes and displays information that is highly relevant to the user's current emotional state. Specifically, upon receiving the user's specification, the server retrieves information from the database and displays it in the application.

[0851] Step 8:

[0852] The user selects the information they need from the displayed search results and completes the purchase process (input). Once the purchase is complete, the server provides the information to the user and awards a reward to the information provider (output). Specifically, the user selects information from the search results and presses the "Purchase" button to complete the payment. The server then sends the purchase information to the user and updates the information provider's account.

[0853] Step 9:

[0854] Once the purchase procedure is completed, the server grants a reward to the information provider (input). This reward is reflected in the information provider's account and can be received as cash or points (output). Specifically, the server updates the information provider's account and adds the amount of the reward or points.

[0855] (Application example 2)

[0856] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0857] Previous systems were unable to take the user's emotions into account when generating useful information from images and location information taken by the user. This resulted in the information provided not necessarily being optimal for the user's current situation and emotions, potentially resulting in a poor user experience. Furthermore, when providing information related to a specific field, optimal recommendations based on the user's emotions and situation were lacking.

[0858] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0859] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for saving the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for analyzing emotions to provide search results related to a specific field, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for providing information related to the field in which the order procedure is included. This makes it possible to provide information optimized for the user's emotions and current situation, thereby improving the user experience.

[0860] An "image" is a photograph or video taken by a user using a mobile device.

[0861] "Location information" is latitude and longitude data used to identify the location where a captured image was taken.

[0862] The "server" is a central processing unit that receives images and location information uploaded by users, analyzes them, and generates information.

[0863] "Useful information" is information that is valuable to the user, and specifically includes information on congestion, availability, weather, and the like.

[0864] The "means for setting a selling price" is an algorithm or system for setting a price for the generated useful information and selling it.

[0865] A "database" is a system for storing useful information that has been generated and managing it so that it can be searched later.

[0866] "Emotion analysis means" is a technology for analyzing the user's emotions and improving the appropriateness of the information provided.

[0867] "Order procedure" refers to a series of procedures that a user must follow to purchase the provided information and complete the transaction.

[0868] An "information provider" is a user who provides useful information by uploading images and location information, and is eligible to receive a reward.

[0869] "Generative AI" is an artificial intelligence technology that analyzes images and location information provided by users and generates useful information.

[0870] System Overview

[0871] This invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and provides that information. In this system, users can easily upload information, and useful information is generated in real time based on that information. It also incorporates emotion analysis means for analyzing user emotions, providing more optimized information. Below, we will explain specific embodiments of this system.

[0872] Required Hardware and Software

[0873] Mobile devices (smartphones, tablets): Used by users to take images and obtain location information.

[0874] GPS function: Used to automatically obtain location information.

[0875] Generative AI models (e.g., GPT-4, DALL-E, etc.): Analyze images and location information and use them to generate useful information.

[0876] Sentiment analysis engine (e.g., Affectiva): Used to analyze user emotions and optimize the information provided.

[0877] A database management system (e.g., MySQL) is used to store and search the useful information generated.

[0878] Cloud server (e.g., AWS): Used to receive data from users and run generative AI models and sentiment analysis engines.

[0879] Detailed System Operation

[0880] The user operates the device to take pictures at a specific location, and the location information is automatically acquired using the GPS function. This data is then uploaded to a cloud server via a dedicated application.

[0881] The cloud server passes the received images and location information to the generative AI model, which then performs image analysis. As a result of the analysis, useful information is generated based on that data. The generated information is then passed to an emotion analysis engine, which analyzes the user's emotions from the uploaded images. Based on the analysis results, the information provided is optimized to the user's current emotions and situation.

[0882] The generated information is stored in a database and made searchable. When a user searches for information, the most relevant information is displayed preferentially based on the results of sentiment analysis. This information is provided when the user completes a purchase, and the information provider is rewarded.

[0883] Specific examples

[0884] Example 1: Combining restaurant congestion and user sentiment

[0885] 1. User: Takes a photo at a restaurant and uploads it to the server through the app.

[0886] 2. Server: Receives images and location information and passes them to the generation AI.

[0887] 3. Generative AI: Analyzes the restaurant's occupancy status and returns the information to the server.

[0888] 4. Emotion Engine: Analyzes user emotions from uploaded images and recommends restaurants with fast service if, for example, you are tired.

[0889] 5. Server: Sets the selling price of the information and stores it in the database. When a user searches, restaurant information that is best suited to the user's tired situation is displayed preferentially.

[0890] Prompt Sentence Examples

[0891] "Please tell me the current congestion situation around Tokyo Tower and recommended food delivery spots."

[0892] This embodiment enables the provision of information optimized according to the user's emotions and circumstances, improving the user experience. Information providers can also easily obtain benefits, and a means is provided for effectively and widely sharing valuable information in real time.

[0893] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0894] Step 1:

[0895] A user takes an image at a specific location using a mobile device. After taking the image, the location information (latitude and longitude) is automatically acquired using the device's GPS function.

[0896] Input: Photo or video, location

[0897] Output: Image data, location data

[0898] Step 2:

[0899] The device uploads the captured images and location information to a cloud server via a dedicated application, making this operation extremely convenient for users.

[0900] Input: Image data, location data

[0901] Output: Data sent to the server

[0902] Step 3:

[0903] The server receives images and location information uploaded by users and passes them to a generative AI model, which uses image analysis algorithms to generate useful information.

[0904] Input: Image data, location data

[0905] Output: Generated useful information (e.g., crowd situation, popular meals)

[0906] Step 4:

[0907] The emotion analysis engine analyzes the user's emotions from the uploaded images, for example, identifying emotions such as joy or fatigue from facial expression analysis and text content.

[0908] Input: Image data

[0909] Output: User emotion data

[0910] Step 5:

[0911] The server receives the generated useful information and the sentiment analysis results, stores the information in a database, and sets a selling price for the information based on a pricing algorithm.

[0912] Input: Generated useful information, user emotion data

[0913] Output: Saved data, set selling price

[0914] Step 6:

[0915] When a user specifies a location using the application's map interface, the server searches the database for registered information near the specified location and displays the results, prioritizing relevant information based on the results of sentiment analysis.

[0916] Input: Search query (location specified), user emotion data

[0917] Output: Search results (information list)

[0918] Step 7:

[0919] The user selects the information they need from the search results and completes the purchase process. The server completes the purchase process and rewards the information provider. The reward is reflected in the information provider's account and can be received as cash or points.

[0920] Input: Purchase procedure information, user account information

[0921] Output: Purchase completion notification, information provider's reward data

[0922] This series of processes allows users to easily obtain useful information based on their own captured images and location information. Furthermore, by combining it with sentiment analysis, it becomes possible to provide information optimized for the user's current situation, significantly improving the user experience. Information providers can also easily benefit from this system, providing a means to effectively share valuable information in real time.

[0923] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0924] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0925] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0926] [Third embodiment]

[0927] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0928] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0929] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0930] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0931] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0932] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0933] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0934] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0935] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0936] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0937] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0938] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0939] The present invention is a system that automatically generates useful information using AI based on images and location information taken by users, and sells that information. Specific embodiments for implementing the present invention are described below.

[0940] User capture and upload of images and location information

[0941] Users use mobile devices such as smartphones and tablets to take photos (or videos) at specific locations. For example, users may take photos at tourist spots, shopping malls, parking lots, etc. When this happens, the device automatically uses its GPS function to obtain location information (latitude and longitude).

[0942] After taking a photo, the user uploads the photo and location information to the server using a dedicated application. The user's operation is extremely simple, and the data can be sent simply by following the application's instructions.

[0943] Server-based data processing and information generation

[0944] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[0945] Road congestion

[0946] Congestion levels at stores and tourist spots

[0947] Parking availability

[0948] Real-time weather information

[0949] Flowering status

[0950] For example, the generative AI can analyze the number of cars and the density of people in an image to determine the congestion level of roads and tourist spots. It can also determine the availability of parking spaces based on the usage status of parking spaces visible in the image. It can also obtain weather information that can be determined from the image.

[0951] The information generated by the AI ​​is returned to the server, which stores it in a database. At the same time, the system also includes a function to automatically set a selling price based on the scarcity and demand of the information.

[0952] Searching for and purchasing information

[0953] When a user wants to search for specific information, they specify the location they want to know about using the application's map interface. For example, if a user wants to know the crowd situation at a particular tourist spot, they specify that point on the map. The server then searches the database for information registered near the specified point and displays the results to the user.

[0954] The user selects the information they need from the displayed results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[0955] Specific examples

[0956] Example 1: Providing information on the congestion status of tourist spots

[0957] 1. User: Takes and uploads a photo at tourist spot A.

[0958] 2. Server: Receives images and location information and passes them to the generation AI.

[0959] 3. Generative AI: Analyzes the level of congestion at tourist spot A and returns the information to the server.

[0960] 4. Server: Sets the selling price of the information and stores it in the database.

[0961] If another user wants to know the congestion situation at tourist spot A,

[0962] 1. User: Specify tourist spot A via the map interface.

[0963] 2. Server: Search and display information about tourist spot A.

[0964] 3. User: Purchases information and obtains congestion status provided by the server.

[0965] Example 2: Providing parking availability information

[0966] 1. User: Takes and uploads a photo in the parking lot.

[0967] 2. Server: Receives images and location information and passes them to the generation AI.

[0968] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[0969] 4. Server: Sets the selling price of the information and stores it in the database.

[0970] If another user wants to know the availability of that parking space,

[0971] 1. User: Specify parking via map interface.

[0972] 2. Server: Searches and displays information about nearby parking lots.

[0973] 3. User: Purchases information and obtains availability provided by the server.

[0974] These embodiments allow users to easily obtain the information they need and provide benefits to information providers. The system provides an efficient means for widely sharing valuable information in real time.

[0975] The processing flow will be explained below.

[0976] Step 1:

[0977] User: Uses a smartphone or tablet to take images (photos or videos) at specific locations, such as tourist spots or parking lots.

[0978] Step 2:

[0979] Device: When an image is taken, the location information (latitude and longitude) is automatically obtained using the GPS function.

[0980] Step 3:

[0981] User: Uploads the captured images and their location information to the server via a dedicated application.

[0982] Step 4:

[0983] Server: Receives images and location information uploaded by users.

[0984] Step 5:

[0985] Server: Passes the received data to the generation AI.

[0986] Step 6:

[0987] Generative AI: Using image analysis algorithms to analyze elements within an image and generate useful information such as:

[0988] Identify road congestion based on the number of cars and people.

[0989] Analyzes the density of people to determine how crowded stores and tourist spots are.

[0990] To identify parking availability based on parking space usage.

[0991] Determine the weather (sunny, cloudy, rainy, etc.) from the photo.

[0992] Analyze the type of flower and its flowering status.

[0993] Step 7:

[0994] Generative AI: Returns the generated useful information to the server.

[0995] Step 8:

[0996] Server: Sets a selling price based on the generated information, taking into account factors such as scarcity and demand for the information.

[0997] Step 9:

[0998] Server: Stores the generated information in a database.

[0999] Step 10:

[1000] Server: Makes the stored information searchable.

[1001] Step 11:

[1002] User (information seeker): Uses the application's map interface to search for information by specifying the location of interest.

[1003] Step 12:

[1004] Server: Searches for information stored in a database near a specified location.

[1005] Step 13:

[1006] Server: Displays the information to the user as search results.

[1007] Step 14:

[1008] User: Select the necessary information from the search results and proceed with the purchase.

[1009] Step 15:

[1010] Server: Provides the user with the relevant information once the purchase is complete.

[1011] Step 16:

[1012] Server: Determines the reward for the informant and reflects it in the account.

[1013] In this way, the system allows users to easily obtain the information they need through specific processing steps, and also allows information providers to easily obtain profits.

[1014] Example 1

[1015] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1016] In modern society, there is a growing demand for useful information obtained in real time. However, existing systems have limitations in the means to efficiently collect, analyze, and provide such information. In particular, systems that combine images and location information to provide information intuitively and immediately have not yet been fully developed. Therefore, an objective of the present invention is to provide a system that automatically generates useful information in real time using images and location information taken by users and makes that information available for sale.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1018] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to a data storage, means for generating useful information in the data storage based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a search system and making it searchable, means for searching for information from the search system and displaying the results, and means for purchasing the searched information and providing a reward to the information provider when the purchase procedure is completed. This allows users to easily obtain useful information in real time, and enables information providers to easily earn profits.

[1019] "Means for adding location information to captured images" refers to a function that uses the camera and GPS functions built into the user's mobile device to automatically add location information such as latitude and longitude to images when they are taken.

[1020] The "means for uploading the images and location information to data storage" refers to a function that allows a user to send the captured images and their location information to a server via a dedicated application and store them in cloud-based data storage.

[1021] The "means for generating useful information based on the images and location information in the data storage" refers to a function that uses a generation AI model in the server to analyze the received images and location information and generate useful information (such as congestion status and parking availability).

[1022] The "means for setting the selling price of the generated information" is an algorithm for evaluating the value of the information generated by the server and automatically calculating an appropriate selling price based on the rarity and demand of the information.

[1023] "Means for storing the generated information in a search system and making it searchable" refers to a function that stores the generated useful information in a database so that users can easily find it through a search system.

[1024] "Means for searching for information from the search system and displaying the results" refers to a function in which, when a user enters specific information into the search system, the server searches for the corresponding information from the database and displays the results on the user's terminal.

[1025] "Means for purchasing searched information and rewarding the information provider once the purchase procedure is completed" refers to a system that manages the payment process for purchasing information found by a user and pays a reward to the user who provided the information once the purchase is completed.

[1026] "Traffic congestion status" refers to information about the density of vehicles and the degree of congestion on specific roads, intersections, etc.

[1027] "Crowding levels at hubs and tourist destinations" refers to information about the density of people and the degree of congestion at specific facilities or areas.

[1028] "Parking area availability" is information about the number and location of available parking spaces in a particular parking lot.

[1029] "Real-time weather information" refers to information about current weather conditions, such as temperature and precipitation, in a particular area.

[1030] "Flowering status of plants" is information about the progress of flowering and the best time to see a particular plant.

[1031] A "generative AI model" is an artificial intelligence algorithm with functions such as image analysis and data generation, and in particular uses computer vision and natural language processing to generate useful information.

[1032] This invention is a system that automatically generates useful information using a generative AI model based on images taken by users and location information, and sells that information. Specific embodiments for implementing the invention are described below.

[1033] Overall system configuration

[1034] Users take photos and videos at specific locations using mobile devices such as smartphones and tablets. At this time, the device uses its GPS function to obtain location information (latitude and longitude). After taking the photos, users upload the images and location information to a server using a dedicated application.

[1035] The server receives images and location information uploaded by users and automatically generates useful information based on that information using a generative AI model. The generated information is stored in a database and made searchable, with a selling price automatically set based on the information's rarity and demand. Finally, when the user searches for and purchases the information and the purchase process is completed, the information provider is rewarded.

[1036] Hardware and software used

[1037] Hardware

[1038] Smartphones and tablets (e.g. iPhone, Android smartphone, iPad)

[1039] Cloud servers (e.g. AWS, Google Cloud, Microsoft Azure)

[1040] software

[1041] Camera app and GPS function on mobile devices

[1042] Dedicated app (iOS app, Android app)

[1043] Generative AI models (e.g., OpenAI GPT-4)

[1044] Database management systems (e.g., MySQL, PostgreSQL)

[1045] Specific details of data processing and calculation

[1046] Images taken by users on their mobile devices and the acquired location information are uploaded to the server via a dedicated app. The server then performs the following processes.

[1047] 1. Data integrity check

[1048] The server checks the consistency of the received image data and location information, checking for errors and omissions.

[1049] 2. Data Preprocessing

[1050] The server converts the image data into a format that can be passed to the generative AI model (e.g., JSON format).

[1051] 3. Analysis and information generation using generative AI models

[1052] The generative AI model analyzes features in an image (e.g., number of cars, human density, weather, etc.) and generates useful information (e.g., congestion level, parking availability, etc.).

[1053] Example of a prompt to be passed to the AI ​​generator:

[1054] "Please analyze the congestion situation at store X from this image."

[1055] "Please identify the availability of this parking space from this image."

[1056] 4. Information Storage and Pricing

[1057] The server stores the generated information in a database and makes it searchable.

[1058] The server automatically sets the selling price based on the scarcity and demand of the information.

[1059] Specific examples

[1060] Example 1: Providing information on the congestion status of tourist spots

[1061] 1. The user takes a photo at tourist spot A and uploads it to the server via a dedicated app.

[1062] 2. The server receives the image and location information and passes it to the generative AI model.

[1063] 3. The generative AI model analyzes the level of congestion at tourist destination A and returns the results to the server.

[1064] 4. The server sets the selling price of the information and stores it in the database.

[1065] If another user wants to know the congestion situation at tourist spot A,

[1066] 1. The user specifies tourist spot A via the map interface.

[1067] 2. The server searches for and displays information about tourist spot A.

[1068] 3. The user purchases information and obtains congestion status provided by the server.

[1069] Example 2: Providing parking availability information

[1070] 1. The user takes a photo in the parking lot and uploads it to the server via a dedicated app.

[1071] 2. The server receives the image and location information and passes it to the generative AI model.

[1072] 3. The generative AI model analyzes the parking availability and returns the results to the server.

[1073] 4. The server sets the selling price of the information and stores it in the database.

[1074] If another user wants to know the availability of that parking space,

[1075] 1. The user specifies a parking spot via the map interface.

[1076] 2. The server searches for and displays information about nearby parking lots.

[1077] 3. The user purchases the information and obtains the availability provided by the server.

[1078] As described above, this system allows users to easily obtain useful information in real time, and information providers can easily benefit from it. This system provides an efficient means for widely sharing valuable information.

[1079] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1080] Step 1:

[1081] User capture and upload of images and location information

[1082] Operation overview

[1083] Input: Images taken by the user on a mobile device and acquired location information

[1084] How it works: The user launches the dedicated application and takes a photo or video at a specific location. At this time, the location information (latitude and longitude) is automatically obtained using the mobile device's GPS function. After taking the photo, the user presses the send button in the dedicated application to upload the image and location information to the server.

[1085] Output: Image data and location information sent to the server

[1086] Step 2:

[1087] Data reception and preprocessing by the server

[1088] Operation overview

[1089] Input: Image data and location information uploaded by the user

[1090] How it works: The server receives an HTTP request and retrieves image and location data. The server checks the data for integrity and errors or omissions. It then converts the image data into an appropriate format (e.g., JSON) to be passed to the generative AI model.

[1091] Output: Preprocessed image data and location information

[1092] Step 3:

[1093] Information generation by generative AI

[1094] Operation overview

[1095] Input: Preprocessed image data and location information

[1096] How it works: The server sends the preprocessed data to a generative AI model. The generative AI model analyzes features in the image (e.g., number of cars, human density, weather, etc.) and automatically generates useful information (e.g., congestion level, parking availability, etc.). Examples of prompts input to the generative AI model include "Please analyze the congestion level at Store X from this image" and "Please identify the availability of this parking lot from this image."

[1097] Output: Useful information as analysis results (such as congestion status and parking availability)

[1098] Step 4:

[1099] Server-based information storage and pricing

[1100] Operation overview

[1101] Input: Useful information from the analysis results obtained from the generative AI model

[1102] How it works: The server stores the generated information in a database. It then applies an algorithm to automatically set the selling price based on the scarcity and demand of the information, and determines an appropriate selling price. The information, along with the selling price, is made searchable.

[1103] Output: Useful information stored in the database and its selling price

[1104] Step 5:

[1105] User search and purchase

[1106] Operation overview

[1107] Input: Search criteria specified by the user (e.g., the congestion status of a specific location)

[1108] How it works: The user specifies the location they want to know about using the map interface in the dedicated app. The server searches the database for information close to the specified location and displays the results to the user. The user selects the information they need from the displayed information and completes the purchase procedure. Once payment is complete, the server provides the information to the user and simultaneously rewards the information provider.

[1109] Output: Useful information provided to the user and the reward given.

[1110] (Application example 1)

[1111] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1112] In modern urban environments, safety management of public areas and private property is a critical issue. In particular, during large-scale events and disasters, it is necessary to grasp safety situations in real time and respond quickly, but current systems are insufficient in this regard. Furthermore, there is no established method for efficiently collecting this safety information and appropriately rewarding data providers. Furthermore, there is a lack of ways for ordinary users to contribute to public safety using their own mobile devices.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1114] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for evaluating the safety of a specific area based on the generated information and issuing an alert. This allows users to easily collect and provide safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[1115] output

[1116] The "means for assigning location information to a captured image" refers to a device or software that provides a function for automatically associating location information such as latitude and longitude with an image captured by a user.

[1117] The "means for uploading the images and location information to the server" refers to a device or software that includes a network connection and communication protocol for transmitting images taken by a user and their associated location information to the server.

[1118] The "means for generating useful information based on the images and location information at the server" refers to an algorithm or computer program that analyzes the received images and location information and generates important data or insights based on specific conditions.

[1119] The "means for setting the selling price of the generated information" is an algorithm or computer program that determines an appropriate selling price based on the market value and demand for the generated useful information.

[1120] "Means for storing the generated information in a database and making it searchable" refers to a database management system that efficiently stores the generated information and makes it easily searchable and accessible later.

[1121] The "means for searching information from the database and displaying the results" refers to an interface and algorithms for extracting relevant information from the database based on a search query from a user and displaying the results to the user.

[1122] "Means for purchasing searched information and rewarding the information provider when the purchase procedure is completed" is a system that automatically awards money or points to the user who provided the information when the transaction is completed after the user purchases the information.

[1123] The "means for assessing the safety of a specific area based on the generated information and issuing an alert" refers to an algorithm and communication device for analyzing the generated information, assessing the safety of a specific area, and issuing an alert to relevant parties as necessary.

[1124] output

[1125] This invention is a system that uses AI to automatically generate useful information based on images and location information taken by users, and provides and sells that information. This system is particularly designed for application in the field of safety management, and is capable of issuing area safety assessments and warnings in real time.

[1126] 1. Acquisition and upload of images and location information by user devices

[1127] Users use mobile devices such as smartphones and tablets to take photos at specific locations. For example, users may take photos in public areas or on private property. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the photo, users upload the photo and location information to a server using a dedicated application. This allows users to easily provide the necessary data to the system.

[1128] 2. Data processing and information generation by the server

[1129] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[1130] Safety assessment of specific areas (detection of suspicious objects, determination of dangerous areas, etc.)

[1131] Real-time information related to public safety (such as congestion and traffic conditions)

[1132] For example, the generating AI analyzes whether an image contains any suspicious objects and uses that information to evaluate the safety of the area. It also evaluates the safety of the surrounding environment in real time based on data obtained from location information. The generated information is returned to the server, which stores it in a database. At the same time, it automatically sets a selling price based on the scarcity and demand of the information.

[1133] 3. Searching for and Purchasing Information

[1134] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. For example, if a user wants to know the safety status of a particular area, they specify that location on the map. The server searches the database for information registered near the specified location and displays the results to the user. The user selects the information they need from the displayed results and proceeds with the purchase. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[1135] Hardware and software used

[1136] Hardware: Smartphones, tablets

[1137] Software: Dedicated application, server, generative AI, Google Cloud Vision API, Google Maps API, requests library, database management system

[1138] Specific examples

[1139] Example 1: Public Area Safety Assessment

[1140] 1. User: Finds a suspicious object in the park, takes a photo, and uploads it. The location information is automatically captured.

[1141] 2. Server: Receives images and location information and passes them to the generation AI.

[1142] 3. Generative AI: Performs a safety assessment of the park and determines that it is a dangerous area.

[1143] 4. Server: Stores the safety assessment results in a database and issues alerts as necessary.

[1144] Prompt Sentence Examples

[1145] "Generate the following information based on the photo and GPS location:

[1146] Is there anything suspicious in the image?

[1147] Assessment of local crime prevention and safety conditions

[1148] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1149] output

[1150] Step 1:

[1151] A user takes a photo at a specific location using a device such as a smartphone or tablet. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). The input is the captured image and its location information. As a result, the image taken by the user is saved along with the location information.

[1152] Step 2:

[1153] After taking a photo, the user uses a dedicated application to upload the photo and location information to the server. The input is the image and location information acquired in step 1. The output is the image and location information stored on the server. The server receives this information and prepares it for the next processing step.

[1154] Step 3:

[1155] The server passes the received image and location information to the generation AI. The input is the image and location information stored on the server. The generation AI uses image analysis algorithms to detect suspicious objects in the uploaded image and evaluate the safety of the area. This analysis is performed by detecting specific objects in the image and matching them with location information. The output is the results of the image analysis (e.g., whether or not there is a suspicious object, the safety evaluation of the area, etc.).

[1156] Step 4:

[1157] The server sets the selling price of the generated information based on the analysis results received from the generation AI. The input is the image analysis results. This automatically determines an appropriate selling price based on market value and demand. The output is the set selling price.

[1158] Step 5:

[1159] The server stores the generated information and its selling price in a database, making it searchable. The input is the generated information and the set selling price. This allows the database to be searchable and accessible by users later, and the output is the information stored in the database.

[1160] Step 6:

[1161] When a user searches for specific information, they use the application's map interface to specify the location they want to know about. The input is the user's search query (e.g., safety status in a particular area). The server searches a database for relevant information based on the query and displays the results. The output is the displayed search results.

[1162] Step 7:

[1163] The user selects the information they need from the displayed search results and completes the purchase process. The input is the selected information and the purchase process information. Once the purchase process is completed, the server provides the information to the user and awards a reward to the information provider. This allows the provider to receive a reward for providing the information, and the user can obtain the information they need. The output is the information provided to the user and the information provider with the award.

[1164] The above processing steps enable users to collect and provide useful safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[1165] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1166] The present invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for implementing the present invention will be described below.

[1167] User capture and upload of images and location information

[1168] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[1169] Server-based data processing and information generation

[1170] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[1171] Road congestion

[1172] Congestion levels at stores and tourist spots

[1173] Parking availability

[1174] Real-time weather information

[1175] Flowering status

[1176] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[1177] Recognizing user emotions with an emotion engine

[1178] The present invention further incorporates an emotion engine. The emotion engine uses technology to recognize a user's emotion by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data contained in images uploaded by the user to identify the user's emotion (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation method of the generated useful information.

[1179] Searching for and purchasing information

[1180] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[1181] Purchase and Use of Information

[1182] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[1183] Specific examples

[1184] Example 1: Combining tourist destination congestion and user sentiment

[1185] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[1186] 2. Server: Receives images and location information and passes them to the generation AI.

[1187] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[1188] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[1189] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[1190] Example 2: Combining parking availability and user sentiment

[1191] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[1192] 2. Server: Receives images and location information and passes them to the generation AI.

[1193] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[1194] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[1195] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[1196] This allows users to easily obtain the information they need in a form optimized for their emotions, and information providers can easily earn profits. This system provides an efficient means for widely sharing valuable information in real time.

[1197] The processing flow will be explained below.

[1198] Step 1:

[1199] User: Take an image (photo or video) with a smartphone or tablet at a specific location (e.g., a tourist spot or parking lot).

[1200] Step 2:

[1201] Device: The location information (latitude and longitude) of the captured image is automatically obtained using the GPS function.

[1202] Step 3:

[1203] User: The captured images and location information are uploaded to the server through a dedicated application. The application has a user-friendly interface and is easy to operate.

[1204] Step 4:

[1205] Server: Receives images and location information uploaded by users and temporarily stores them in a database.

[1206] Step 5:

[1207] Server: Passes the received image and location information to the generation AI, which runs an image analysis algorithm to analyze elements within the image.

[1208] Step 6:

[1209] Generative AI: Analyzes elements in an image and generates useful information. Examples include:

[1210] Analyze the number of cars and people to identify road congestion.

[1211] Analyzes the density of people to determine how crowded stores and tourist spots are.

[1212] Analyze parking space usage to identify available parking spaces.

[1213] Determine the weather (sunny, cloudy, rainy, etc.) from the image.

[1214] Analyze the type of flower and its flowering status.

[1215] Step 7:

[1216] Generative AI: Returns the generated useful information to the server.

[1217] Step 8:

[1218] Server: Sets a selling price based on the useful information received, taking into account factors such as scarcity and demand for the information.

[1219] Step 9:

[1220] Server: Passes the generated information to the emotion engine and provides the information necessary for analyzing user emotions.

[1221] Step 10:

[1222] Emotion engine: Analyzes image and text content to recognize the user's emotional state (e.g., joy, sadness, surprise, etc.).

[1223] Step 11:

[1224] Emotion engine: Optimizes the presentation of generated information based on the user's emotional state, for example, suggesting less congested routes for users who are feeling stressed.

[1225] Step 12:

[1226] Server: Based on the results of the emotion engine, the optimized information is stored in a database and made searchable.

[1227] Step 13:

[1228] User (information searcher): Uses the map interface of the application to search for information by specifying the location they want to know about. For example, if a user wants to know the congestion status of a tourist spot, they specify that location on the map.

[1229] Step 14:

[1230] Server: Searches the database for registered information near the specified location and displays the results to the user.

[1231] Step 15:

[1232] User: Selects the information they need from the search results and completes the purchase. They also check the information presented based on their emotional state.

[1233] Step 16:

[1234] Server: Once the purchase process is complete, the server provides the relevant information to the user and simultaneously initiates the reward process for the information provider.

[1235] Step 17:

[1236] Server: Determines the reward for the informant's account and reflects it in cash or points.

[1237] In this way, through specific processing steps, the user can easily obtain the information they need in a form optimized for their emotions, and the information provider can easily obtain profits from the system.

[1238] Example 2

[1239] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1240] Current information provision systems can generate useful information based on images and location information provided by users, but this information is often provided without considering the user's emotional state. This makes it difficult to provide optimal information tailored to the user's needs and circumstances. Furthermore, the selling price of the generated information is not optimized, and rewards to information providers are not efficiently provided. These issues need to be resolved.

[1241] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for assigning location information to a captured image, a means for uploading the image and location information to the server, a means for using a generation AI in the server to generate useful information based on the image and location information, a means for setting a selling price for the generated information, a means for saving the generated information in a database and making it searchable, a means for searching for information from the database and displaying the results, a means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and a means for using an emotion engine that recognizes emotions from the image and location information uploaded by the user. This makes it possible to provide optimal information according to the user's emotional state and efficiently set the selling price of the information. Furthermore, it also becomes possible to efficiently reward information providers.

[1242] "Photographed images" are photos or videos taken by a user using a mobile device such as a smartphone or tablet.

[1243] "Location information" refers to latitude and longitude data obtained using the GPS function of a mobile device.

[1244] A "server" is a computer system that has the ability to receive, process, store, retrieve, and display data.

[1245] "Generative AI" is an artificial intelligence model that uses image analysis algorithms to automatically generate useful information from images and location information.

[1246] "Useful information" is information that is valuable to users, such as road congestion, congestion at shopping facilities and tourist spots, parking availability, real-time weather information, and the blooming status of plants.

[1247] The "sale price" is the price at which the generated useful information is offered to the user.

[1248] A "database" is an information management system that stores generated information and keeps it searchable.

[1249] The "emotion engine" is an analytical engine that recognizes a user's emotions from images and location information.

[1250] An "information provider" is a user who contributes to the generation of useful information by providing captured images and location information.

[1251] "Upload" means sending image and location information data from the user's mobile terminal to the server.

[1252] This invention is a system that automatically generates useful information using a generation AI based on images and location information taken by users, sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for carrying out the invention are described below.

[1253] User capture and upload of images and location information

[1254] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[1255] Server-based data processing and information generation

[1256] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[1257] Road congestion

[1258] Congestion levels at sales facilities and tourist spots

[1259] Parking availability

[1260] Real-time weather information

[1261] Flowering status of plants

[1262] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[1263] Recognizing user emotions with an emotion engine

[1264] The server is also equipped with an emotion engine. The emotion engine uses technology to recognize a user's emotions by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data included in images uploaded by the user to identify the user's emotions (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation of the generated useful information.

[1265] Searching for and purchasing information

[1266] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[1267] Purchase and Use of Information

[1268] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[1269] Specific examples

[1270] Example 1: Combining tourist destination congestion and user sentiment

[1271] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[1272] 2. Server: Receives images and location information and passes them to the generation AI.

[1273] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[1274] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[1275] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[1276] Example 2: Combining parking availability and user sentiment

[1277] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[1278] 2. Server: Receives images and location information and passes them to the generation AI.

[1279] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[1280] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[1281] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[1282] Prompt Sentence Examples

[1283] "Please tell me about the congestion situation at tourist spot A. Please also take user emotions into consideration when displaying this information."

[1284] "Provide optimal information based on parking availability and user sentiment."

[1285] This system provides an efficient means for widely sharing valuable information in real time, and can improve the user experience by providing the most appropriate information according to the user's emotions.

[1286] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1287] Step 1:

[1288] A user uses a smartphone or tablet to take an image (photo or video) at a specific location. When taking a photo, the device automatically uses its GPS function to obtain location information (latitude and longitude). The captured image (input) and the obtained location information (input) are uploaded to a server (output) via a dedicated application. Specifically, the user presses the "take a photo" button in the application, and then presses the "upload" button after taking the photo.

[1289] Step 2:

[1290] The server receives the image and location information uploaded by the user (input). The server temporarily stores the received data and verifies that the image and location information were received correctly (output). Specifically, the server receives the HTTP request and analyzes the image file and location information.

[1291] Step 3:

[1292] The server sends the received image and location information to the generating AI (input). This transmission is done by calling the generating AI's API (output). Specifically, the server executes an HTTP POST request to send data to the generating AI model's endpoint.

[1293] Step 4:

[1294] The generating AI analyzes the received images and generates useful information based on the location information (input). The generated useful information (output) includes road congestion, the congestion level of sales facilities, parking availability, real-time weather information, and the blooming status of plants. Specifically, the generating AI applies image analysis algorithms to generate various types of information.

[1295] Step 5:

[1296] After receiving the information returned by the generation AI, the server executes the emotion engine (input). The emotion engine analyzes the facial expressions and voice data in the image to identify the user's emotional state (output). Specifically, the emotion engine recognizes the facial expressions in the image and determines the user's emotions, such as whether they are happy or sad.

[1297] Step 6:

[1298] The server stores the generated information and the user's emotional state in a database (input) and keeps it searchable (output). At the same time, it uses a pricing algorithm to set a selling price for the generated information. Specifically, the server sets a price based on the type and demand of the generated information and stores the information in the database.

[1299] Step 7:

[1300] When a user specifies a specific location using the application's map interface (input), the server searches the database for registered information about the area around that location and displays it to the user (output). Based on the emotion engine, the server prioritizes and displays information that is highly relevant to the user's current emotional state. Specifically, upon receiving the user's specification, the server retrieves information from the database and displays it in the application.

[1301] Step 8:

[1302] The user selects the information they need from the displayed search results and completes the purchase process (input). Once the purchase is complete, the server provides the information to the user and awards a reward to the information provider (output). Specifically, the user selects information from the search results and presses the "Purchase" button to complete the payment. The server then sends the purchase information to the user and updates the information provider's account.

[1303] Step 9:

[1304] Once the purchase procedure is completed, the server grants a reward to the information provider (input). This reward is reflected in the information provider's account and can be received as cash or points (output). Specifically, the server updates the information provider's account and adds the amount of the reward or points.

[1305] (Application example 2)

[1306] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1307] Previous systems were unable to take the user's emotions into account when generating useful information from images and location information taken by the user. This resulted in the information provided not necessarily being optimal for the user's current situation and emotions, potentially resulting in a poor user experience. Furthermore, when providing information related to a specific field, optimal recommendations based on the user's emotions and situation were lacking.

[1308] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1309] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for saving the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for analyzing emotions to provide search results related to a specific field, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for providing information related to the field in which the order procedure is included. This makes it possible to provide information optimized for the user's emotions and current situation, thereby improving the user experience.

[1310] An "image" is a photograph or video taken by a user using a mobile device.

[1311] "Location information" is latitude and longitude data used to identify the location where a captured image was taken.

[1312] The "server" is a central processing unit that receives images and location information uploaded by users, analyzes them, and generates information.

[1313] "Useful information" is information that is valuable to the user, and specifically includes information on congestion, availability, weather, and the like.

[1314] The "means for setting a selling price" is an algorithm or system for setting a price for the generated useful information and selling it.

[1315] A "database" is a system for storing useful information that has been generated and managing it so that it can be searched later.

[1316] "Emotion analysis means" is a technology for analyzing the user's emotions and improving the appropriateness of the information provided.

[1317] "Order procedure" refers to a series of procedures that a user must follow to purchase the provided information and complete the transaction.

[1318] An "information provider" is a user who provides useful information by uploading images and location information, and is eligible to receive a reward.

[1319] "Generative AI" is an artificial intelligence technology that analyzes images and location information provided by users and generates useful information.

[1320] System Overview

[1321] This invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and provides that information. In this system, users can easily upload information, and useful information is generated in real time based on that information. It also incorporates emotion analysis means for analyzing user emotions, providing more optimized information. Below, we will explain specific embodiments of this system.

[1322] Required Hardware and Software

[1323] Mobile devices (smartphones, tablets): Used by users to take images and obtain location information.

[1324] GPS function: Used to automatically obtain location information.

[1325] Generative AI models (e.g., GPT-4, DALL-E, etc.): Analyze images and location information and use them to generate useful information.

[1326] Sentiment analysis engine (e.g., Affectiva): Used to analyze user emotions and optimize the information provided.

[1327] A database management system (e.g., MySQL) is used to store and search the useful information generated.

[1328] Cloud server (e.g., AWS): Used to receive data from users and run generative AI models and sentiment analysis engines.

[1329] Detailed System Operation

[1330] The user operates the device to take pictures at a specific location, and the location information is automatically acquired using the GPS function. This data is then uploaded to a cloud server via a dedicated application.

[1331] The cloud server passes the received images and location information to the generative AI model, which then performs image analysis. As a result of the analysis, useful information is generated based on that data. The generated information is then passed to an emotion analysis engine, which analyzes the user's emotions from the uploaded images. Based on the analysis results, the information provided is optimized to the user's current emotions and situation.

[1332] The generated information is stored in a database and made searchable. When a user searches for information, the most relevant information is displayed preferentially based on the results of sentiment analysis. This information is provided when the user completes a purchase, and the information provider is rewarded.

[1333] Specific examples

[1334] Example 1: Combining restaurant congestion and user sentiment

[1335] 1. User: Takes a photo at a restaurant and uploads it to the server through the app.

[1336] 2. Server: Receives images and location information and passes them to the generation AI.

[1337] 3. Generative AI: Analyzes the restaurant's occupancy status and returns the information to the server.

[1338] 4. Emotion Engine: Analyzes user emotions from uploaded images and recommends restaurants with fast service if, for example, you are tired.

[1339] 5. Server: Sets the selling price of the information and stores it in the database. When a user searches, restaurant information that is best suited to the user's tired situation is displayed preferentially.

[1340] Prompt Sentence Examples

[1341] "Please tell me the current congestion situation around Tokyo Tower and recommended food delivery spots."

[1342] This embodiment enables the provision of information optimized according to the user's emotions and circumstances, improving the user experience. Information providers can also easily obtain benefits, and a means is provided for effectively and widely sharing valuable information in real time.

[1343] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1344] Step 1:

[1345] A user takes an image at a specific location using a mobile device. After taking the image, the location information (latitude and longitude) is automatically acquired using the device's GPS function.

[1346] Input: Photo or video, location

[1347] Output: Image data, location data

[1348] Step 2:

[1349] The device uploads the captured images and location information to a cloud server via a dedicated application, making this operation extremely convenient for users.

[1350] Input: Image data, location data

[1351] Output: Data sent to the server

[1352] Step 3:

[1353] The server receives images and location information uploaded by users and passes them to a generative AI model, which uses image analysis algorithms to generate useful information.

[1354] Input: Image data, location data

[1355] Output: Generated useful information (e.g., crowd situation, popular meals)

[1356] Step 4:

[1357] The emotion analysis engine analyzes the user's emotions from the uploaded images, for example, identifying emotions such as joy or fatigue from facial expression analysis and text content.

[1358] Input: Image data

[1359] Output: User emotion data

[1360] Step 5:

[1361] The server receives the generated useful information and the sentiment analysis results, stores the information in a database, and sets a selling price for the information based on a pricing algorithm.

[1362] Input: Generated useful information, user emotion data

[1363] Output: Saved data, set selling price

[1364] Step 6:

[1365] When a user specifies a location using the application's map interface, the server searches the database for registered information near the specified location and displays the results, prioritizing relevant information based on the results of sentiment analysis.

[1366] Input: Search query (location specified), user emotion data

[1367] Output: Search results (information list)

[1368] Step 7:

[1369] The user selects the information they need from the search results and completes the purchase process. The server completes the purchase process and rewards the information provider. The reward is reflected in the information provider's account and can be received as cash or points.

[1370] Input: Purchase procedure information, user account information

[1371] Output: Purchase completion notification, information provider's reward data

[1372] This series of processes allows users to easily obtain useful information based on their own captured images and location information. Furthermore, by combining it with sentiment analysis, it becomes possible to provide information optimized for the user's current situation, significantly improving the user experience. Information providers can also easily benefit from this system, providing a means to effectively share valuable information in real time.

[1373] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1374] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1375] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1376] [Fourth embodiment]

[1377] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1378] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1379] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1380] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1381] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1382] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1383] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1384] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1385] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1386] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1387] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1388] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1389] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1390] The present invention is a system that automatically generates useful information using AI based on images and location information taken by users, and sells that information. Specific embodiments for implementing the present invention are described below.

[1391] User capture and upload of images and location information

[1392] Users use mobile devices such as smartphones and tablets to take photos (or videos) at specific locations. For example, users may take photos at tourist spots, shopping malls, parking lots, etc. When this happens, the device automatically uses its GPS function to obtain location information (latitude and longitude).

[1393] After taking a photo, the user uploads the photo and location information to the server using a dedicated application. The user's operation is extremely simple, and the data can be sent simply by following the application's instructions.

[1394] Server-based data processing and information generation

[1395] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[1396] Road congestion

[1397] Congestion levels at stores and tourist spots

[1398] Parking availability

[1399] Real-time weather information

[1400] Flowering status

[1401] For example, the generative AI can analyze the number of cars and the density of people in an image to determine the congestion level of roads and tourist spots. It can also determine the availability of parking spaces based on the usage status of parking spaces visible in the image. It can also obtain weather information that can be determined from the image.

[1402] The information generated by the AI ​​is returned to the server, which stores it in a database. At the same time, the system also includes a function to automatically set a selling price based on the scarcity and demand of the information.

[1403] Searching for and purchasing information

[1404] When a user wants to search for specific information, they specify the location they want to know about using the application's map interface. For example, if a user wants to know the crowd situation at a particular tourist spot, they specify that point on the map. The server then searches the database for information registered near the specified point and displays the results to the user.

[1405] The user selects the information they need from the displayed results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[1406] Specific examples

[1407] Example 1: Providing information on the congestion status of tourist spots

[1408] 1. User: Takes and uploads a photo at tourist spot A.

[1409] 2. Server: Receives images and location information and passes them to the generation AI.

[1410] 3. Generative AI: Analyzes the level of congestion at tourist spot A and returns the information to the server.

[1411] 4. Server: Sets the selling price of the information and stores it in the database.

[1412] If another user wants to know the congestion situation at tourist spot A,

[1413] 1. User: Specify tourist spot A via the map interface.

[1414] 2. Server: Search and display information about tourist spot A.

[1415] 3. User: Purchases information and obtains congestion status provided by the server.

[1416] Example 2: Providing parking availability information

[1417] 1. User: Takes and uploads a photo in the parking lot.

[1418] 2. Server: Receives images and location information and passes them to the generation AI.

[1419] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[1420] 4. Server: Sets the selling price of the information and stores it in the database.

[1421] If another user wants to know the availability of that parking space,

[1422] 1. User: Specify parking via map interface.

[1423] 2. Server: Searches and displays information about nearby parking lots.

[1424] 3. User: Purchases information and obtains availability provided by the server.

[1425] These embodiments allow users to easily obtain the information they need and provide benefits to information providers. The system provides an efficient means for widely sharing valuable information in real time.

[1426] The processing flow will be explained below.

[1427] Step 1:

[1428] User: Uses a smartphone or tablet to take images (photos or videos) at specific locations, such as tourist spots or parking lots.

[1429] Step 2:

[1430] Device: When an image is taken, the location information (latitude and longitude) is automatically obtained using the GPS function.

[1431] Step 3:

[1432] User: Uploads the captured images and their location information to the server via a dedicated application.

[1433] Step 4:

[1434] Server: Receives images and location information uploaded by users.

[1435] Step 5:

[1436] Server: Passes the received data to the generation AI.

[1437] Step 6:

[1438] Generative AI: Using image analysis algorithms to analyze elements within an image and generate useful information such as:

[1439] Identify road congestion based on the number of cars and people.

[1440] Analyzes the density of people to determine how crowded stores and tourist spots are.

[1441] To identify parking availability based on parking space usage.

[1442] Determine the weather (sunny, cloudy, rainy, etc.) from the photo.

[1443] Analyze the type of flower and its flowering status.

[1444] Step 7:

[1445] Generative AI: Returns the generated useful information to the server.

[1446] Step 8:

[1447] Server: Sets a selling price based on the generated information, taking into account factors such as scarcity and demand for the information.

[1448] Step 9:

[1449] Server: Stores the generated information in a database.

[1450] Step 10:

[1451] Server: Makes the stored information searchable.

[1452] Step 11:

[1453] User (information seeker): Uses the application's map interface to search for information by specifying the location of interest.

[1454] Step 12:

[1455] Server: Searches for information stored in a database near a specified location.

[1456] Step 13:

[1457] Server: Displays the information to the user as search results.

[1458] Step 14:

[1459] User: Select the necessary information from the search results and proceed with the purchase.

[1460] Step 15:

[1461] Server: Provides the user with the relevant information once the purchase is complete.

[1462] Step 16:

[1463] Server: Determines the reward for the informant and reflects it in the account.

[1464] In this way, the system allows users to easily obtain the information they need through specific processing steps, and also allows information providers to easily obtain profits.

[1465] Example 1

[1466] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1467] In modern society, there is a growing demand for useful information obtained in real time. However, existing systems have limitations in the means to efficiently collect, analyze, and provide such information. In particular, systems that combine images and location information to provide information intuitively and immediately have not yet been fully developed. Therefore, an objective of the present invention is to provide a system that automatically generates useful information in real time using images and location information taken by users and makes that information available for sale.

[1468] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1469] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to a data storage, means for generating useful information in the data storage based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a search system and making it searchable, means for searching for information from the search system and displaying the results, and means for purchasing the searched information and providing a reward to the information provider when the purchase procedure is completed. This allows users to easily obtain useful information in real time, and enables information providers to easily earn profits.

[1470] "Means for adding location information to captured images" refers to a function that uses the camera and GPS functions built into the user's mobile device to automatically add location information such as latitude and longitude to images when they are taken.

[1471] The "means for uploading the images and location information to data storage" refers to a function that allows a user to send the captured images and their location information to a server via a dedicated application and store them in cloud-based data storage.

[1472] The "means for generating useful information based on the images and location information in the data storage" refers to a function that uses a generation AI model in the server to analyze the received images and location information and generate useful information (such as congestion status and parking availability).

[1473] The "means for setting the selling price of the generated information" is an algorithm for evaluating the value of the information generated by the server and automatically calculating an appropriate selling price based on the rarity and demand of the information.

[1474] "Means for storing the generated information in a search system and making it searchable" refers to a function that stores the generated useful information in a database so that users can easily find it through a search system.

[1475] "Means for searching for information from the search system and displaying the results" refers to a function in which, when a user enters specific information into the search system, the server searches for the corresponding information from the database and displays the results on the user's terminal.

[1476] "Means for purchasing searched information and rewarding the information provider once the purchase procedure is completed" refers to a system that manages the payment process for purchasing information found by a user and pays a reward to the user who provided the information once the purchase is completed.

[1477] "Traffic congestion status" refers to information about the density of vehicles and the degree of congestion on specific roads, intersections, etc.

[1478] "Crowding levels at hubs and tourist destinations" refers to information about the density of people and the degree of congestion at specific facilities or areas.

[1479] "Parking area availability" is information about the number and location of available parking spaces in a particular parking lot.

[1480] "Real-time weather information" refers to information about current weather conditions, such as temperature and precipitation, in a particular area.

[1481] "Flowering status of plants" is information about the progress of flowering and the best time to see a particular plant.

[1482] A "generative AI model" is an artificial intelligence algorithm with functions such as image analysis and data generation, and in particular uses computer vision and natural language processing to generate useful information.

[1483] This invention is a system that automatically generates useful information using a generative AI model based on images taken by users and location information, and sells that information. Specific embodiments for implementing the invention are described below.

[1484] Overall system configuration

[1485] Users take photos and videos at specific locations using mobile devices such as smartphones and tablets. At this time, the device uses its GPS function to obtain location information (latitude and longitude). After taking the photos, users upload the images and location information to a server using a dedicated application.

[1486] The server receives images and location information uploaded by users and automatically generates useful information based on that information using a generative AI model. The generated information is stored in a database and made searchable, with a selling price automatically set based on the information's rarity and demand. Finally, when the user searches for and purchases the information and the purchase process is completed, the information provider is rewarded.

[1487] Hardware and software used

[1488] Hardware

[1489] Smartphones and tablets (e.g. iPhone, Android smartphone, iPad)

[1490] Cloud servers (e.g. AWS, Google Cloud, Microsoft Azure)

[1491] software

[1492] Camera app and GPS function on mobile devices

[1493] Dedicated app (iOS app, Android app)

[1494] Generative AI models (e.g., OpenAI GPT-4)

[1495] Database management systems (e.g., MySQL, PostgreSQL)

[1496] Specific details of data processing and calculation

[1497] Images taken by users on their mobile devices and the acquired location information are uploaded to the server via a dedicated app. The server then performs the following processes.

[1498] 1. Data integrity check

[1499] The server checks the consistency of the received image data and location information, checking for errors and omissions.

[1500] 2. Data Preprocessing

[1501] The server converts the image data into a format that can be passed to the generative AI model (e.g., JSON format).

[1502] 3. Analysis and information generation using generative AI models

[1503] The generative AI model analyzes features in an image (e.g., number of cars, human density, weather, etc.) and generates useful information (e.g., congestion level, parking availability, etc.).

[1504] Example of a prompt to be passed to the AI ​​generator:

[1505] "Please analyze the congestion situation at store X from this image."

[1506] "Please identify the availability of this parking space from this image."

[1507] 4. Information Storage and Pricing

[1508] The server stores the generated information in a database and makes it searchable.

[1509] The server automatically sets the selling price based on the scarcity and demand of the information.

[1510] Specific examples

[1511] Example 1: Providing information on the congestion status of tourist spots

[1512] 1. The user takes a photo at tourist spot A and uploads it to the server via a dedicated app.

[1513] 2. The server receives the image and location information and passes it to the generative AI model.

[1514] 3. The generative AI model analyzes the level of congestion at tourist destination A and returns the results to the server.

[1515] 4. The server sets the selling price of the information and stores it in the database.

[1516] If another user wants to know the congestion situation at tourist spot A,

[1517] 1. The user specifies tourist spot A via the map interface.

[1518] 2. The server searches for and displays information about tourist spot A.

[1519] 3. The user purchases information and obtains congestion status provided by the server.

[1520] Example 2: Providing parking availability information

[1521] 1. The user takes a photo in the parking lot and uploads it to the server via a dedicated app.

[1522] 2. The server receives the image and location information and passes it to the generative AI model.

[1523] 3. The generative AI model analyzes the parking availability and returns the results to the server.

[1524] 4. The server sets the selling price of the information and stores it in the database.

[1525] If another user wants to know the availability of that parking space,

[1526] 1. The user specifies a parking spot via the map interface.

[1527] 2. The server searches for and displays information about nearby parking lots.

[1528] 3. The user purchases the information and obtains the availability provided by the server.

[1529] As described above, this system allows users to easily obtain useful information in real time, and information providers can easily benefit from it. This system provides an efficient means for widely sharing valuable information.

[1530] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1531] Step 1:

[1532] User capture and upload of images and location information

[1533] Operation overview

[1534] Input: Images taken by the user on a mobile device and acquired location information

[1535] How it works: The user launches the dedicated application and takes a photo or video at a specific location. At this time, the location information (latitude and longitude) is automatically obtained using the mobile device's GPS function. After taking the photo, the user presses the send button in the dedicated application to upload the image and location information to the server.

[1536] Output: Image data and location information sent to the server

[1537] Step 2:

[1538] Data reception and preprocessing by the server

[1539] Operation overview

[1540] Input: Image data and location information uploaded by the user

[1541] How it works: The server receives an HTTP request and retrieves image and location data. The server checks the data for integrity and errors or omissions. It then converts the image data into an appropriate format (e.g., JSON) to be passed to the generative AI model.

[1542] Output: Preprocessed image data and location information

[1543] Step 3:

[1544] Information generation by generative AI

[1545] Operation overview

[1546] Input: Preprocessed image data and location information

[1547] How it works: The server sends the preprocessed data to a generative AI model. The generative AI model analyzes features in the image (e.g., number of cars, human density, weather, etc.) and automatically generates useful information (e.g., congestion level, parking availability, etc.). Examples of prompts input to the generative AI model include "Please analyze the congestion level at Store X from this image" and "Please identify the availability of this parking lot from this image."

[1548] Output: Useful information as analysis results (such as congestion status and parking availability)

[1549] Step 4:

[1550] Server-based information storage and pricing

[1551] Operation overview

[1552] Input: Useful information from the analysis results obtained from the generative AI model

[1553] How it works: The server stores the generated information in a database. It then applies an algorithm to automatically set the selling price based on the scarcity and demand of the information, and determines an appropriate selling price. The information, along with the selling price, is made searchable.

[1554] Output: Useful information stored in the database and its selling price

[1555] Step 5:

[1556] User search and purchase

[1557] Operation overview

[1558] Input: Search criteria specified by the user (e.g., the congestion status of a specific location)

[1559] How it works: The user specifies the location they want to know about using the map interface in the dedicated app. The server searches the database for information close to the specified location and displays the results to the user. The user selects the information they need from the displayed information and completes the purchase procedure. Once payment is complete, the server provides the information to the user and simultaneously rewards the information provider.

[1560] Output: Useful information provided to the user and the reward given.

[1561] (Application example 1)

[1562] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1563] In modern urban environments, safety management of public areas and private property is a critical issue. In particular, during large-scale events and disasters, it is necessary to grasp safety situations in real time and respond quickly, but current systems are insufficient in this regard. Furthermore, there is no established method for efficiently collecting this safety information and appropriately rewarding data providers. Furthermore, there is a lack of ways for ordinary users to contribute to public safety using their own mobile devices.

[1564] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1565] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for storing the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for evaluating the safety of a specific area based on the generated information and issuing an alert. This allows users to easily collect and provide safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[1566] output

[1567] The "means for assigning location information to a captured image" refers to a device or software that provides a function for automatically associating location information such as latitude and longitude with an image captured by a user.

[1568] The "means for uploading the images and location information to the server" refers to a device or software that includes a network connection and communication protocol for transmitting images taken by a user and their associated location information to the server.

[1569] The "means for generating useful information based on the images and location information at the server" refers to an algorithm or computer program that analyzes the received images and location information and generates important data or insights based on specific conditions.

[1570] The "means for setting the selling price of the generated information" is an algorithm or computer program that determines an appropriate selling price based on the market value and demand for the generated useful information.

[1571] "Means for storing the generated information in a database and making it searchable" refers to a database management system that efficiently stores the generated information and makes it easily searchable and accessible later.

[1572] The "means for searching information from the database and displaying the results" refers to an interface and algorithms for extracting relevant information from the database based on a search query from a user and displaying the results to the user.

[1573] "Means for purchasing searched information and rewarding the information provider when the purchase procedure is completed" is a system that automatically awards money or points to the user who provided the information when the transaction is completed after the user purchases the information.

[1574] The "means for assessing the safety of a specific area based on the generated information and issuing an alert" refers to an algorithm and communication device for analyzing the generated information, assessing the safety of a specific area, and issuing an alert to relevant parties as necessary.

[1575] output

[1576] This invention is a system that uses AI to automatically generate useful information based on images and location information taken by users, and provides and sells that information. This system is particularly designed for application in the field of safety management, and is capable of issuing area safety assessments and warnings in real time.

[1577] 1. Acquisition and upload of images and location information by user devices

[1578] Users use mobile devices such as smartphones and tablets to take photos at specific locations. For example, users may take photos in public areas or on private property. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the photo, users upload the photo and location information to a server using a dedicated application. This allows users to easily provide the necessary data to the system.

[1579] 2. Data processing and information generation by the server

[1580] The server receives images and location information uploaded by users. The received data is first passed to the generation AI, which uses image analysis algorithms to automatically generate useful information such as:

[1581] Safety assessment of specific areas (detection of suspicious objects, determination of dangerous areas, etc.)

[1582] Real-time information related to public safety (such as congestion and traffic conditions)

[1583] For example, the generating AI analyzes whether an image contains any suspicious objects and uses that information to evaluate the safety of the area. It also evaluates the safety of the surrounding environment in real time based on data obtained from location information. The generated information is returned to the server, which stores it in a database. At the same time, it automatically sets a selling price based on the scarcity and demand of the information.

[1584] 3. Searching for and Purchasing Information

[1585] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. For example, if a user wants to know the safety status of a particular area, they specify that location on the map. The server searches the database for information registered near the specified location and displays the results to the user. The user selects the information they need from the displayed results and proceeds with the purchase. Once the purchase is complete, the server provides the information to the user and simultaneously awards a reward to the information provider. This reward is recorded in the information provider's account and reflected in the form of cash or points.

[1586] Hardware and software used

[1587] Hardware: Smartphones, tablets

[1588] Software: Dedicated application, server, generative AI, Google Cloud Vision API, Google Maps API, requests library, database management system

[1589] Specific examples

[1590] Example 1: Public Area Safety Assessment

[1591] 1. User: Finds a suspicious object in the park, takes a photo, and uploads it. The location information is automatically captured.

[1592] 2. Server: Receives images and location information and passes them to the generation AI.

[1593] 3. Generative AI: Performs a safety assessment of the park and determines that it is a dangerous area.

[1594] 4. Server: Stores the safety assessment results in a database and issues alerts as necessary.

[1595] Prompt Sentence Examples

[1596] "Generate the following information based on the photo and GPS location:

[1597] Is there anything suspicious in the image?

[1598] Assessment of local crime prevention and safety conditions

[1599] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1600] output

[1601] Step 1:

[1602] A user takes a photo at a specific location using a device such as a smartphone or tablet. At this time, the device automatically uses its GPS function to obtain location information (latitude and longitude). The input is the captured image and its location information. As a result, the image taken by the user is saved along with the location information.

[1603] Step 2:

[1604] After taking a photo, the user uses a dedicated application to upload the photo and location information to the server. The input is the image and location information acquired in step 1. The output is the image and location information stored on the server. The server receives this information and prepares it for the next processing step.

[1605] Step 3:

[1606] The server passes the received image and location information to the generation AI. The input is the image and location information stored on the server. The generation AI uses image analysis algorithms to detect suspicious objects in the uploaded image and evaluate the safety of the area. This analysis is performed by detecting specific objects in the image and matching them with location information. The output is the results of the image analysis (e.g., whether or not there is a suspicious object, the safety evaluation of the area, etc.).

[1607] Step 4:

[1608] The server sets the selling price of the generated information based on the analysis results received from the generation AI. The input is the image analysis results. This automatically determines an appropriate selling price based on market value and demand. The output is the set selling price.

[1609] Step 5:

[1610] The server stores the generated information and its selling price in a database, making it searchable. The input is the generated information and the set selling price. This allows the database to be searchable and accessible by users later, and the output is the information stored in the database.

[1611] Step 6:

[1612] When a user searches for specific information, they use the application's map interface to specify the location they want to know about. The input is the user's search query (e.g., safety status in a particular area). The server searches a database for relevant information based on the query and displays the results. The output is the displayed search results.

[1613] Step 7:

[1614] The user selects the information they need from the displayed search results and completes the purchase process. The input is the selected information and the purchase process information. Once the purchase process is completed, the server provides the information to the user and awards a reward to the information provider. This allows the provider to receive a reward for providing the information, and the user can obtain the information they need. The output is the information provided to the user and the information provider with the award.

[1615] The above processing steps enable users to collect and provide useful safety information in real time, enabling prompt responses and appropriate rewards to information providers.

[1616] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1617] The present invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for implementing the present invention will be described below.

[1618] User capture and upload of images and location information

[1619] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[1620] Server-based data processing and information generation

[1621] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[1622] Road congestion

[1623] Congestion levels at stores and tourist spots

[1624] Parking availability

[1625] Real-time weather information

[1626] Flowering status

[1627] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[1628] Recognizing user emotions with an emotion engine

[1629] The present invention further incorporates an emotion engine. The emotion engine uses technology to recognize a user's emotion by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data contained in images uploaded by the user to identify the user's emotion (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation method of the generated useful information.

[1630] Searching for and purchasing information

[1631] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[1632] Purchase and Use of Information

[1633] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[1634] Specific examples

[1635] Example 1: Combining tourist destination congestion and user sentiment

[1636] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[1637] 2. Server: Receives images and location information and passes them to the generation AI.

[1638] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[1639] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[1640] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[1641] Example 2: Combining parking availability and user sentiment

[1642] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[1643] 2. Server: Receives images and location information and passes them to the generation AI.

[1644] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[1645] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[1646] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[1647] This allows users to easily obtain the information they need in a form optimized for their emotions, and information providers can easily earn profits. This system provides an efficient means for widely sharing valuable information in real time.

[1648] The processing flow will be explained below.

[1649] Step 1:

[1650] User: Take an image (photo or video) with a smartphone or tablet at a specific location (e.g., a tourist spot or parking lot).

[1651] Step 2:

[1652] Device: The location information (latitude and longitude) of the captured image is automatically obtained using the GPS function.

[1653] Step 3:

[1654] User: The captured images and location information are uploaded to the server through a dedicated application. The application has a user-friendly interface and is easy to operate.

[1655] Step 4:

[1656] Server: Receives images and location information uploaded by users and temporarily stores them in a database.

[1657] Step 5:

[1658] Server: Passes the received image and location information to the generation AI, which runs an image analysis algorithm to analyze elements within the image.

[1659] Step 6:

[1660] Generative AI: Analyzes elements in an image and generates useful information. Examples include:

[1661] Analyze the number of cars and people to identify road congestion.

[1662] Analyzes the density of people to determine how crowded stores and tourist spots are.

[1663] Analyze parking space usage to identify available parking spaces.

[1664] Determine the weather (sunny, cloudy, rainy, etc.) from the image.

[1665] Analyze the type of flower and its flowering status.

[1666] Step 7:

[1667] Generative AI: Returns the generated useful information to the server.

[1668] Step 8:

[1669] Server: Sets a selling price based on the useful information received, taking into account factors such as scarcity and demand for the information.

[1670] Step 9:

[1671] Server: Passes the generated information to the emotion engine and provides the information necessary for analyzing user emotions.

[1672] Step 10:

[1673] Emotion engine: Analyzes image and text content to recognize the user's emotional state (e.g., joy, sadness, surprise, etc.).

[1674] Step 11:

[1675] Emotion engine: Optimizes the presentation of generated information based on the user's emotional state, for example, suggesting less congested routes for users who are feeling stressed.

[1676] Step 12:

[1677] Server: Based on the results of the emotion engine, the optimized information is stored in a database and made searchable.

[1678] Step 13:

[1679] User (information searcher): Uses the map interface of the application to search for information by specifying the location they want to know about. For example, if a user wants to know the congestion status of a tourist spot, they specify that location on the map.

[1680] Step 14:

[1681] Server: Searches the database for registered information near the specified location and displays the results to the user.

[1682] Step 15:

[1683] User: Selects the information they need from the search results and completes the purchase. They also check the information presented based on their emotional state.

[1684] Step 16:

[1685] Server: Once the purchase process is complete, the server provides the relevant information to the user and simultaneously initiates the reward process for the information provider.

[1686] Step 17:

[1687] Server: Determines the reward for the informant's account and reflects it in cash or points.

[1688] In this way, through specific processing steps, the user can easily obtain the information they need in a form optimized for their emotions, and the information provider can easily obtain profits from the system.

[1689] Example 2

[1690] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1691] Current information provision systems can generate useful information based on images and location information provided by users, but this information is often provided without considering the user's emotional state. This makes it difficult to provide optimal information tailored to the user's needs and circumstances. Furthermore, the selling price of the generated information is not optimized, and rewards to information providers are not efficiently provided. These issues need to be resolved.

[1692] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for assigning location information to a captured image, a means for uploading the image and location information to the server, a means for using a generation AI in the server to generate useful information based on the image and location information, a means for setting a selling price for the generated information, a means for saving the generated information in a database and making it searchable, a means for searching for information from the database and displaying the results, a means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and a means for using an emotion engine that recognizes emotions from the image and location information uploaded by the user. This makes it possible to provide optimal information according to the user's emotional state and efficiently set the selling price of the information. Furthermore, it also becomes possible to efficiently reward information providers.

[1693] "Photographed images" are photos or videos taken by a user using a mobile device such as a smartphone or tablet.

[1694] "Location information" refers to latitude and longitude data obtained using the GPS function of a mobile device.

[1695] A "server" is a computer system that has the ability to receive, process, store, retrieve, and display data.

[1696] "Generative AI" is an artificial intelligence model that uses image analysis algorithms to automatically generate useful information from images and location information.

[1697] "Useful information" is information that is valuable to users, such as road congestion, congestion at shopping facilities and tourist spots, parking availability, real-time weather information, and the blooming status of plants.

[1698] The "sale price" is the price at which the generated useful information is offered to the user.

[1699] A "database" is an information management system that stores generated information and keeps it searchable.

[1700] The "emotion engine" is an analytical engine that recognizes a user's emotions from images and location information.

[1701] An "information provider" is a user who contributes to the generation of useful information by providing captured images and location information.

[1702] "Upload" means sending image and location information data from the user's mobile terminal to the server.

[1703] This invention is a system that automatically generates useful information using a generation AI based on images and location information taken by users, sells that information, and also recognizes the user's emotions by combining it with an emotion engine. Specific embodiments for carrying out the invention are described below.

[1704] User capture and upload of images and location information

[1705] Users use a mobile device such as a smartphone or tablet to take an image (photo or video) at a specific location. The device automatically uses its GPS function to obtain location information (latitude and longitude). After taking the image, the user uploads the photo and location information to a server using a dedicated application. This operation is very simple for users.

[1706] Server-based data processing and information generation

[1707] The server receives the images and location information uploaded by the user and passes that data to the Generator AI, which uses image analysis algorithms to generate useful information such as:

[1708] Road congestion

[1709] Congestion levels at sales facilities and tourist spots

[1710] Parking availability

[1711] Real-time weather information

[1712] Flowering status of plants

[1713] The generated information is returned to the server, which uses a pricing algorithm to set a selling price for the information, and then stores the generated information in a database where it can be searched.

[1714] Recognizing user emotions with an emotion engine

[1715] The server is also equipped with an emotion engine. The emotion engine uses technology to recognize a user's emotions by analyzing image and text content. Specifically, it analyzes facial expressions and tone of voice data included in images uploaded by the user to identify the user's emotions (e.g., joy, sadness, surprise, etc.). This emotion information is used to optimize the presentation of the generated useful information.

[1716] Searching for and purchasing information

[1717] When a user wants to search for specific information, they use the application's map interface to specify the location they want to know about. The server then searches the database for registered information near the specified location and displays the results. The emotion engine adjusts the display to prioritize more relevant information based on the user's current emotional state. For example, if a user wants to avoid crowded tourist spots, information about less crowded places will be prioritized.

[1718] Purchase and Use of Information

[1719] The user selects the information they need from the displayed search results and completes the purchase process. Once the purchase is complete, the server provides the information to the user and rewards the information provider. This reward is reflected in the information provider's account and can be received in the form of cash or points.

[1720] Specific examples

[1721] Example 1: Combining tourist destination congestion and user sentiment

[1722] 1. User: Takes a photo at tourist spot A and uploads it to the server via the app.

[1723] 2. Server: Receives images and location information and passes them to the generation AI.

[1724] 3. Generative AI: Analyzes the congestion situation at tourist spot A and returns the results to the server.

[1725] 4. Emotion engine: Analyzes user emotions from uploaded images (e.g., determining whether they are having fun).

[1726] 5. Server: Sets the selling price of the information and stores it in the database. Considers the user's emotional state and displays relevant information appropriately.

[1727] Example 2: Combining parking availability and user sentiment

[1728] 1. User: Takes a photo in the parking lot and uploads it to the server via the app.

[1729] 2. Server: Receives images and location information and passes them to the generation AI.

[1730] 3. Generative AI: Analyzes parking availability and returns the information to the server.

[1731] 4. Emotion Engine: Analyzes emotions from uploaded images or user text input to determine whether the user is feeling stressed.

[1732] 5. Server: Set the selling price of the information and save it in the database. If the user is stressed, display the nearest available parking lot information first.

[1733] Prompt Sentence Examples

[1734] "Please tell me about the congestion situation at tourist spot A. Please also take user emotions into consideration when displaying this information."

[1735] "Provide optimal information based on parking availability and user sentiment."

[1736] This system provides an efficient means for widely sharing valuable information in real time, and can improve the user experience by providing the most appropriate information according to the user's emotions.

[1737] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1738] Step 1:

[1739] A user uses a smartphone or tablet to take an image (photo or video) at a specific location. When taking a photo, the device automatically uses its GPS function to obtain location information (latitude and longitude). The captured image (input) and the obtained location information (input) are uploaded to a server (output) via a dedicated application. Specifically, the user presses the "take a photo" button in the application, and then presses the "upload" button after taking the photo.

[1740] Step 2:

[1741] The server receives the image and location information uploaded by the user (input). The server temporarily stores the received data and verifies that the image and location information were received correctly (output). Specifically, the server receives the HTTP request and analyzes the image file and location information.

[1742] Step 3:

[1743] The server sends the received image and location information to the generating AI (input). This transmission is done by calling the generating AI's API (output). Specifically, the server executes an HTTP POST request to send data to the generating AI model's endpoint.

[1744] Step 4:

[1745] The generating AI analyzes the received images and generates useful information based on the location information (input). The generated useful information (output) includes road congestion, the congestion level of sales facilities, parking availability, real-time weather information, and the blooming status of plants. Specifically, the generating AI applies image analysis algorithms to generate various types of information.

[1746] Step 5:

[1747] After receiving the information returned by the generation AI, the server executes the emotion engine (input). The emotion engine analyzes the facial expressions and voice data in the image to identify the user's emotional state (output). Specifically, the emotion engine recognizes the facial expressions in the image and determines the user's emotions, such as whether they are happy or sad.

[1748] Step 6:

[1749] The server stores the generated information and the user's emotional state in a database (input) and keeps it searchable (output). At the same time, it uses a pricing algorithm to set a selling price for the generated information. Specifically, the server sets a price based on the type and demand of the generated information and stores the information in the database.

[1750] Step 7:

[1751] When a user specifies a specific location using the application's map interface (input), the server searches the database for registered information about the area around that location and displays it to the user (output). Based on the emotion engine, the server prioritizes and displays information that is highly relevant to the user's current emotional state. Specifically, upon receiving the user's specification, the server retrieves information from the database and displays it in the application.

[1752] Step 8:

[1753] The user selects the information they need from the displayed search results and completes the purchase process (input). Once the purchase is complete, the server provides the information to the user and awards a reward to the information provider (output). Specifically, the user selects information from the search results and presses the "Purchase" button to complete the payment. The server then sends the purchase information to the user and updates the information provider's account.

[1754] Step 9:

[1755] Once the purchase procedure is completed, the server grants a reward to the information provider (input). This reward is reflected in the information provider's account and can be received as cash or points (output). Specifically, the server updates the information provider's account and adds the amount of the reward or points.

[1756] (Application example 2)

[1757] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1758] Previous systems were unable to take the user's emotions into account when generating useful information from images and location information taken by the user. This resulted in the information provided not necessarily being optimal for the user's current situation and emotions, potentially resulting in a poor user experience. Furthermore, when providing information related to a specific field, optimal recommendations based on the user's emotions and situation were lacking.

[1759] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1760] In this invention, the server includes means for assigning location information to captured images, means for uploading the images and location information to the server, means for generating useful information in the server based on the images and location information, means for setting a selling price for the generated information, means for saving the generated information in a database and making it searchable, means for searching for information from the database and displaying the results, means for analyzing emotions to provide search results related to a specific field, means for purchasing the searched information and rewarding the information provider when the purchase procedure is completed, and means for providing information related to the field in which the order procedure is included. This makes it possible to provide information optimized for the user's emotions and current situation, thereby improving the user experience.

[1761] An "image" is a photograph or video taken by a user using a mobile device.

[1762] "Location information" is latitude and longitude data used to identify the location where a captured image was taken.

[1763] The "server" is a central processing unit that receives images and location information uploaded by users, analyzes them, and generates information.

[1764] "Useful information" is information that is valuable to the user, and specifically includes information on congestion, availability, weather, and the like.

[1765] The "means for setting a selling price" is an algorithm or system for setting a price for the generated useful information and selling it.

[1766] A "database" is a system for storing useful information that has been generated and managing it so that it can be searched later.

[1767] "Emotion analysis means" is a technology for analyzing the user's emotions and improving the appropriateness of the information provided.

[1768] "Order procedure" refers to a series of procedures that a user must follow to purchase the provided information and complete the transaction.

[1769] An "information provider" is a user who provides useful information by uploading images and location information, and is eligible to receive a reward.

[1770] "Generative AI" is an artificial intelligence technology that analyzes images and location information provided by users and generates useful information.

[1771] System Overview

[1772] This invention is a system that uses a generation AI to automatically generate useful information based on images and location information taken by users, and provides that information. In this system, users can easily upload information, and useful information is generated in real time based on that information. It also incorporates emotion analysis means for analyzing user emotions, providing more optimized information. Below, we will explain specific embodiments of this system.

[1773] Required Hardware and Software

[1774] Mobile devices (smartphones, tablets): Used by users to take images and obtain location information.

[1775] GPS function: Used to automatically obtain location information.

[1776] Generative AI models (e.g., GPT-4, DALL-E, etc.): Analyze images and location information and use them to generate useful information.

[1777] Sentiment analysis engine (e.g., Affectiva): Used to analyze user emotions and optimize the information provided.

[1778] A database management system (e.g., MySQL) is used to store and search the useful information generated.

[1779] Cloud server (e.g., AWS): Used to receive data from users and run generative AI models and sentiment analysis engines.

[1780] Detailed System Operation

[1781] The user operates the device to take pictures at a specific location, and the location information is automatically acquired using the GPS function. This data is then uploaded to a cloud server via a dedicated application.

[1782] The cloud server passes the received images and location information to the generative AI model, which then performs image analysis. As a result of the analysis, useful information is generated based on that data. The generated information is then passed to an emotion analysis engine, which analyzes the user's emotions from the uploaded images. Based on the analysis results, the information provided is optimized to the user's current emotions and situation.

[1783] The generated information is stored in a database and made searchable. When a user searches for information, the most relevant information is displayed preferentially based on the results of sentiment analysis. This information is provided when the user completes a purchase, and the information provider is rewarded.

[1784] Specific examples

[1785] Example 1: Combining restaurant congestion and user sentiment

[1786] 1. User: Takes a photo at a restaurant and uploads it to the server through the app.

[1787] 2. Server: Receives images and location information and passes them to the generation AI.

[1788] 3. Generative AI: Analyzes the restaurant's occupancy status and returns the information to the server.

[1789] 4. Emotion Engine: Analyzes user emotions from uploaded images and recommends restaurants with fast service if, for example, you are tired.

[1790] 5. Server: Sets the selling price of the information and stores it in the database. When a user searches, restaurant information that is best suited to the user's tired situation is displayed preferentially.

[1791] Prompt Sentence Examples

[1792] "Please tell me the current congestion situation around Tokyo Tower and recommended food delivery spots."

[1793] This embodiment enables the provision of information optimized according to the user's emotions and circumstances, improving the user experience. Information providers can also easily obtain benefits, and a means is provided for effectively and widely sharing valuable information in real time.

[1794] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1795] Step 1:

[1796] A user takes an image at a specific location using a mobile device. After taking the image, the location information (latitude and longitude) is automatically acquired using the device's GPS function.

[1797] Input: Photo or video, location

[1798] Output: Image data, location data

[1799] Step 2:

[1800] The device uploads the captured images and location information to a cloud server via a dedicated application, making this operation extremely convenient for users.

[1801] Input: Image data, location data

[1802] Output: Data sent to the server

[1803] Step 3:

[1804] The server receives images and location information uploaded by users and passes them to a generative AI model, which uses image analysis algorithms to generate useful information.

[1805] Input: Image data, location data

[1806] Output: Generated useful information (e.g., crowd situation, popular meals)

[1807] Step 4:

[1808] The emotion analysis engine analyzes the user's emotions from the uploaded images, for example, identifying emotions such as joy or fatigue from facial expression analysis and text content.

[1809] Input: Image data

[1810] Output: User emotion data

[1811] Step 5:

[1812] The server receives the generated useful information and the sentiment analysis results, stores the information in a database, and sets a selling price for the information based on a pricing algorithm.

[1813] Input: Generated useful information, user emotion data

[1814] Output: Saved data, set selling price

[1815] Step 6:

[1816] When a user specifies a location using the application's map interface, the server searches the database for registered information near the specified location and displays the results, prioritizing relevant information based on the results of sentiment analysis.

[1817] Input: Search query (location specified), user emotion data

[1818] Output: Search results (information list)

[1819] Step 7:

[1820] The user selects the information they need from the search results and completes the purchase process. The server completes the purchase process and rewards the information provider. The reward is reflected in the information provider's account and can be received as cash or points.

[1821] Input: Purchase procedure information, user account information

[1822] Output: Purchase completion notification, information provider's reward data

[1823] This series of processes allows users to easily obtain useful information based on their own captured images and location information. Furthermore, by combining it with sentiment analysis, it becomes possible to provide information optimized for the user's current situation, significantly improving the user experience. Information providers can also easily benefit from this system, providing a means to effectively share valuable information in real time.

[1824] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1825] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1826] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1827] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1828] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1829] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1830] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1831] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1832] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1833] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1834] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1835] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1836] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1837] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1838] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1839] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1840] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1841] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1842] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1843] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1844] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1845] The following is further disclosed regarding the above embodiment.

[1846] (Claim 1)

[1847] a means for adding location information to the captured image;

[1848] means for uploading the image and location information to a server;

[1849] a means for generating useful information based on the image and location information in a server;

[1850] means for setting a selling price for the generated information;

[1851] means for storing the generated information in a database and making it searchable;

[1852] means for retrieving information from said database and displaying the results;

[1853] A means for purchasing the searched information and giving a reward to the information provider when the purchase procedure is completed;

[1854] Including system.

[1855] (Claim 2)

[1856] 2. The system according to claim 1, wherein the useful information is any one of road congestion, congestion at stores or tourist spots, parking availability, real-time weather information, and flower blooming status.

[1857] (Claim 3)

[1858] The system of claim 1, further comprising a generative AI that analyzes the image.

[1859] "Example 1"

[1860] (Claim 1)

[1861] a means for adding location information to the captured image;

[1862] means for uploading said image and location information to a data storage;

[1863] means for generating useful information based on the image and location information in a data storage; 【1...

Claims

1. a means for adding location information to the captured image; means for uploading the image and location information to a server; a means for generating useful information based on the image and location information in a server; means for setting a selling price for the generated information; means for storing the generated information in a database and making it searchable; means for retrieving information from said database and displaying the results; A means for purchasing the searched information and giving a reward to the information provider when the purchase procedure is completed; Including system.

2. 2. The system according to claim 1, wherein the useful information is any one of road congestion, congestion at stores or tourist spots, parking availability, real-time weather information, and flower blooming status.

3. The system of claim 1 further comprising a generative AI for analyzing the image.

Images