system

Abstract

We provide the system. [Solution] Means for receiving image data and location information captured by the user, A generating device that analyzes the aforementioned image data and location information to generate information about a specific situation, A means for automatically setting the price for providing the generated information based on its scarcity, Information provision means that stores the aforementioned information in a storage device and allows the user to search for and purchase it, A means for processing payment to the information provider when the aforementioned information is purchased, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern information transmission, the problems encountered by users are particularly in situations that require real-time performance. In conventional information sharing methods, if the information provider does not use appropriate keywords or explanations, it is difficult for users seeking the required information to find it. In addition, in many cases, the mechanism of how valuable the provided information is specifically and how the information provider can obtain remuneration according to its value is not clear. Therefore, the valuation of beneficial information and its smooth circulation have been hindered.

Means for Solving the Problems

[0005] This invention provides a generation device that receives image data and location information captured by users, analyzes them, and generates specific situational information such as traffic conditions, congestion levels, and weather information. Furthermore, the generated information's rarity is evaluated, and a price is automatically set. The information is stored in a memory device, searchable and purchasable by other users, and upon successful purchase, the information provider is paid a reward using electronic payment methods. In this way, a system is realized that effectively converts real-time, useful information into value and facilitates its smooth distribution among users.

[0006] A "user" is an individual or organization that uses the system to provide image data and location information, and consumes or purchases the information obtained.

[0007] "Image data" refers to digital data containing visual information captured by a user using a device.

[0008] "Location information" refers to information that indicates a specific location using GPS or other technologies, and is described as geographical coordinates.

[0009] A "generation device" is a device or system that analyzes image data and location information to generate specific situational information.

[0010] "Scarcity" is a factor that evaluates the uniqueness and rarity of generated information and influences its value.

[0011] A "memory device" is a device that has the function of temporarily or permanently storing generated information.

[0012] "Electronic payment methods" refer to methods of exchanging money using digital technology, and include bank transfers and electronic wallets.

[0013] "Reward" refers to the monetary compensation given to a user for the information they provide.

[0014] "Analysis" is a process of performing processing based on image data and position information and extracting meaningful information.

[0015] "Information providing means" is a function that presents the generated information in a form accessible to the user and enables searching and purchasing.

Brief Explanation of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of a data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of a data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention relates to a system implemented via a smartphone or mobile device used by a user on a daily basis. The system begins with the user transmitting image data captured with the device and its location information to a server. The transmitted data is analyzed by a generation device on the server to generate specific situational information.

[0038] Image data captured by users will be obtained by combining a smartphone camera application with location services. Users can easily take images of a location using their device and automatically link the situation and geographical location to the image. For example, it could be used to inform other users about the congestion level at a tourist destination that a user has visited.

[0039] When the server receives data sent from a terminal, it first uses an image analysis algorithm based on the image data to identify individual elements (people, vehicles, sky conditions, etc.). This allows for estimations of congestion levels at tourist spots and theme parks, road traffic conditions, or weather conditions.

[0040] Location information, when combined with analysis results, is organized as local information for a specific region. This allows for the measurement of information scarcity and the evaluation of its value and demand to users. The server then sets the selling price of the information based on this scarcity data, but the selling price fluctuates depending on market demand and the availability of other similar information.

[0041] The generated information is stored in the server's storage and then made available for other users to access. Other users can use their devices to search for this information on the app and purchase it at an appropriate price. Once the purchase is complete, the server automatically processes the payment to the information provider via electronic payment. This system facilitates the smooth provision of data on a daily basis and its subsequent economic value creation.

[0042] This invention is particularly effective in realizing services targeting users residing in urban areas who are frequently on the move and have a high demand for real-time information, as well as in providing information related to the tourism industry.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The user activates their smartphone and launches a dedicated application. Within the app, the user uses the camera function to take pictures of specific locations or objects. The device simultaneously uses its GPS function to collect location information at the time of photo capture.

[0046] Step 2:

[0047] The device uploads the captured image data and location information to the server. The data is transmitted securely using the HTTPS protocol. The device notifies the user that the transmission was successful.

[0048] Step 3:

[0049] The server receives image data and location information transmitted from the terminal. The server checks the data structure and format to ensure that there are no missing or incorrect data. Once validation is complete, the process moves on to analysis.

[0050] Step 4:

[0051] A generator on the server analyzes the image data and uses image recognition algorithms to identify elements in the photograph (e.g., people, cars, buildings, sky, etc.). The analysis results generate specific situational information (e.g., congestion level, traffic conditions, weather, etc.).

[0052] Step 5:

[0053] The server associates generated situational information with location information and evaluates the rarity of the information. Based on this rarity, the server automatically determines the selling price of the information. The price is set considering market demand and the availability of other similar information.

[0054] Step 6:

[0055] The server stores the generated information in a database and prepares it for other users to access and purchase later. This information can be searched using maps and keywords within the application.

[0056] Step 7:

[0057] The user (the user searching for information) uses the application to search for the information they need and select the information that interests them. The user reviews the details of the information and, if they wish to purchase it, proceeds with the purchase process.

[0058] Step 8:

[0059] Once the server confirms that the information has been purchased, it begins the process of paying the user who provided the information. The payment is made using the registered electronic payment method. This allows the information provider to receive economic compensation commensurate with the information they provided.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In today's technological society, there is a demand for the efficient collection of highly accurate information in real time and the provision of that information as economic value. However, existing information provision systems have challenges such as limited user participation in information gathering and difficulty in setting prices based on the scarcity and market value of the information. Furthermore, a process for appropriately compensating information providers has not been established. This invention aims to solve these problems and provide a system that effectively connects the stages of information collection, analysis, and provision.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for processing image data and location information received via data equipment, an analysis device for analyzing the image data and location information and generating specific situational information, and means for automatically determining the price of the information based on the market evaluation of the generated situational information. This makes it possible to efficiently collect local information with user participation, quickly provide highly valuable information, and pay appropriate compensation to information providers.

[0065] A "data device" is an electronic device equipped with image capture and communication functions used by users to input information and transmit it to a server.

[0066] "Image data" refers to a digital representation of visual information captured by a user and transmitted through a data device.

[0067] "Location information" refers to geographical coordinate information acquired by data devices and transmitted along with image data.

[0068] "Means of processing" refers to the procedures or systems that allow the server to receive the received image data and location information and convert it into an analyzable format.

[0069] "Analysis" is data processing that uses algorithms based on image data and location information to identify specific environments or situations.

[0070] An "analysis device" is a part of a system that generates situational information from image data and location information using algorithms such as generative AI models.

[0071] "Situational information" refers to generated information that indicates the local conditions at a specific location, based on analyzed image data and location information.

[0072] "Market valuation" is the process of evaluating the value of generated situational information based on its scarcity and demand.

[0073] "Means for automatically determining the offering price" refers to a system function for setting the selling price of information based on market evaluation.

[0074] This invention is a system for efficiently collecting and utilizing information. Users take photographs of local conditions using a smartphone or mobile device, and the device's location information function acquires geographic coordinates during this process. The software used consists of a camera application and location services on the smartphone.

[0075] Image data and location information captured by the user are transmitted to the server via the device. Data transmission is securely performed using encrypted communication protocols (e.g., HTTPS).

[0076] The server analyzes the received image data using a generative AI model. This analysis process recognizes people, vehicles, and environmental elements (such as weather) within the image, and generates local situation information based on this recognition. Specific software used includes image analysis libraries and AI frameworks.

[0077] The generated status information is stored on the server, and pricing is automatically set according to market value. This pricing is dynamically adjusted by an algorithm that reflects market valuation.

[0078] Other users can access the server through an application on their device and purchase information they are interested in. When information is purchased, the purchase process is carried out through an electronic payment system, and once the purchase is complete, the server pays a reward to the information provider. This reward is processed through PayPal or bank APIs.

[0079] For example, if a user takes a photo of the crowd situation at a tourist spot and sends the information to the server, and that information is purchased by another user, the user who provided the information will be paid a reward.

[0080] Example prompt: "Please provide the latest information to assess the current crowd situation and weather at Kiyomizu-dera Temple, a popular tourist destination."

[0081] In this way, the system collects real-world information in real time and maximizes its value.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The user takes a picture of the location using their smartphone's camera application. At this time, location services are activated, and latitude and longitude information is linked to the image data. The input is the image taken by the user and the GPS location information, and the output is a dataset combining these two. This dataset is sent from the device to the server.

[0085] Step 2:

[0086] The server receives the transmitted dataset. The processing performed here involves storing and initially formatting the image data and location information. Specifically, the image data is converted into a parseable format, and the location information is processed into text format. In addition, data integrity checks are performed to enhance the reliability of the transmission.

[0087] Step 3:

[0088] The server performs image analysis on the received dataset using a generative AI model. It receives image data as input and uses an analysis algorithm to identify elements and environmental information (people, vehicles, weather, etc.) within the image. The output is situational information containing the identified information. This generates specific details about the location, such as congestion levels and weather conditions.

[0089] Step 4:

[0090] The server automatically sets prices for the generated situational information based on market valuations. Specifically, it stores the situational information in a database and applies an algorithm that determines the price by referencing similar historical data and market demand. The output of this process is the situational information with a sales price assigned to it.

[0091] Step 5:

[0092] Other users access the server using a smartphone app to search for and purchase situational information of interest. The input is the user's search query, and the server provides a list of information with prices as output. The purchase process then proceeds, and the transaction is completed using electronic payment methods.

[0093] Step 6:

[0094] Once a user completes their purchase of information, the server pays the information provider. Specifically, electronic payment is made via PayPal or a bank API based on the payment information. The inputs are purchase information and payment information, and the output is a notification that the payment has been completed. This allows information providers to receive compensation for the data they provide.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] In modern urban life, food delivery services are rapidly expanding, but delivery partners face unpredictable factors such as traffic congestion and weather changes, making fast and efficient deliveries difficult. Therefore, optimizing delivery routes and sharing real-time status information are essential.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes a device for receiving image data and location information captured by a user, a device for analyzing the image data and location information and generating information related to a specific situation, and a function for analyzing situation data along the delivery route and providing an optimized delivery route. This enables delivery partners to understand local traffic conditions and weather in real time and make deliveries using more efficient routes.

[0100] A "user" refers to an individual or legal entity that utilizes information or services, and is the entity that takes image data and provides location information.

[0101] "Image data" refers to visual information captured by a user through a device, and serves as basic data for analyzing specific situations.

[0102] "Location information" refers to information that indicates the geographical coordinates where the image data in question was taken, and is used to understand the situation in a specific area.

[0103] A "generation device" refers to a device or system that analyzes received image data and location information to generate specific situational information.

[0104] "Scarcity" is a criterion for evaluating the value and demand of information based on factors such as the frequency of information being provided and the existence of similar information.

[0105] The "information provision function" refers to a function that stores generated information and makes it available for other users to search for or purchase.

[0106] "Compensation" refers to the payment made by an information provider for information they sell to other users, and it is either monetary or of equivalent value.

[0107] "Traffic conditions" refers to information about factors that affect travel, such as road congestion, traffic jams, and traffic volume.

[0108] "Delivery route optimization" refers to adjusting delivery routes to reach the destination in the shortest time or at the lowest cost.

[0109] This system is based on users collecting environmental information using smart devices and analyzing it on a server. Users efficiently acquire image data and location information using devices such as smartphones. Specifically, it utilizes the camera and location information services (for example, Android®'s FusedLocationProviderClient and iOS's Core Location) built into the smartphone. When a user takes a picture at a designated location, the image data is automatically acquired and sent to the server along with the current location information.

[0110] The server receives transmitted image data and location information and processes this data using image analysis algorithms such as OpenCV. For example, it recognizes traffic conditions, congestion, and weather conditions from the image data and performs analysis in conjunction with the location information. Based on these analysis results, information is generated in real time, and prices are automatically set according to the rarity of the information.

[0111] The server then stores the generated information in cloud storage, allowing users to search for and purchase the information as needed through the application. This system automatically pays information providers electronically for the information they purchase.

[0112] As a further example, if a delivery partner records traffic congestion or weather changes during their delivery, the app can capture these scenes, analyze them in real time, and suggest the best route for other delivery partners. An example of a prompt message in this case would be, "Analyze the images showing traffic congestion and optimize your delivery route."

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] The user takes photos of their surroundings using a smart device. The input consists of image data and location information captured by the device's camera. The device uses location services to add the current geographic coordinates to the captured images. The output is a dataset combining the image data and location information.

[0116] Step 2:

[0117] The device sends the acquired image data and location information to the server. The input is a dataset captured by the user. This is securely transmitted to the server using protocols such as HTTPS. The output is the image data and location information received by the server.

[0118] Step 3:

[0119] The server performs image analysis using the received data. The input is image data received from the terminal. The server uses libraries such as OpenCV and TENSORFLOW® to analyze features in the image and extract information such as traffic conditions, weather, and congestion. The output is situational information generated as a result of the analysis.

[0120] Step 4:

[0121] The server combines the situational information obtained through analysis with location information to assess the rarity of the information. The inputs are the situational information and location information generated in step 3. The server applies an algorithm to assess rarity and calculates the market value of the information. The output is the price at which the information is offered.

[0122] Step 5:

[0123] The server stores the generated information in a database, making it accessible to users. The input is information containing the value proposition calculated in step 4. The server registers this information in a cloud-based database. The output is the stored information, which can be searched and retrieved by other users.

[0124] Step 6:

[0125] When a user purchases information, the server processes the payment and pays the information provider. The inputs are the user's purchase request and payment information. The server uses an electronic payment service to execute the transaction and distribute the reward to the information provider. The outputs are a transaction completion notification and the payment of the reward.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention provides an information generation system incorporating emotion recognition technology, accessible via a mobile device typically used by the user. This system is characterized by its ability to analyze the user's emotions using an emotion engine, in addition to image data and location information captured by the user with a smartphone or similar device. As a result, the generated information can reflect the emotions the user felt at that moment.

[0128] The user first launches a dedicated application and uses its camera function to capture images of the site. At this time, the device automatically acquires location information and sends this information to the server. Furthermore, based on the captured images, the device analyzes the user's emotions via an emotion engine and sends the results to the server as additional data.

[0129] When the server receives this data, a generating device analyzes the image data and location information to generate information about a specific situation. For example, it can generate information that integrates not only the congestion level of a tourist spot but also the emotions of the people there (such as excitement, surprise, or boredom). This emotional information is integrated as an element that enhances the value of the situational information.

[0130] Furthermore, the server has an algorithm that evaluates the value of the generated information based on scarcity and sentiment data, and automatically sets the selling price. If the user's sentiment is positive, the information's value may increase, and as a result, the selling price of the information will be adjusted.

[0131] The generated information is stored in a database and later made accessible to other users. Users who need the information can search for it through the application based on geographical conditions and keywords, and then choose to purchase it. Once a purchase is completed, the server pays the user who provided the information through an electronic payment method.

[0132] As an example, when a user visits a theme park on a holiday and sends images of scenery and events taken at the park through the app, the emotion engine analyzes their level of enjoyment and satisfaction and adds this information. Other users can obtain this information through the app, allowing them to understand not only the detailed situation at the visited location but also the genuine emotions of the people there. By providing real-time information including emotion data in this way, information purchasers can make higher-quality decisions.

[0133] This invention is expected to be particularly useful in the tourism industry, event management, and other industries where real-time feedback is beneficial.

[0134] The following describes the processing flow.

[0135] Step 1:

[0136] The user launches a dedicated application on their smartphone and uses the photo-taking function to capture images of the site. Simultaneously, the device utilizes its GPS function to collect precise location information.

[0137] Step 2:

[0138] The device runs an emotion engine based on the captured image, analyzing the user's facial expressions and other image elements to estimate the user's emotions. The estimated emotion data is packaged together with the image data and location information.

[0139] Step 3:

[0140] The device sends the packaged data to the server. For security reasons, data transmission is performed via the HTTPS protocol. Users can confirm within the app that the data has been successfully uploaded.

[0141] Step 4:

[0142] The server analyzes the received image data, location information, and sentiment data. Based on this information, the generator produces specific situational information (for example, the level of crowding at a theme park and the level of enjoyment of visitors there).

[0143] Step 5:

[0144] The server combines generated situational information with sentiment data to assess its rarity and calculate its economic value. The selling price of the information is automatically set based on its rarity and the intensity of the sentiment.

[0145] Step 6:

[0146] The server stores the evaluated information in a database, making it accessible to other users through the app. This process ensures that the information is properly categorized and designed to be searchable.

[0147] Step 7:

[0148] Users who need information can search for it within the application and select the information that suits their purpose. After reviewing the details of the selected information, they can proceed with the purchase.

[0149] Step 8:

[0150] Once the purchase of information is complete, the server initiates the process of paying the user who provided the information. Payment is made using a pre-registered electronic payment method. This process ensures that the information provider receives payment for their services.

[0151] (Example 2)

[0152] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0153] In modern information delivery systems, there is a problem in that the information provided by users is merely a collection of data and fails to adequately reflect the atmosphere and emotional value of the moment. In particular, in tourist destinations and events, there is a demand for information that includes the atmosphere and emotions of the place in real time, but conventional systems have had difficulty achieving this. In addition, the pricing of the generated information often does not accurately reflect supply and demand, and the payment of appropriate compensation to users has not been smooth.

[0154] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0155] In this invention, the server includes means for receiving image information, location data, and emotion data captured by a user; a generation device for analyzing the image information, location data, and emotion data and generating comprehensive information about a specific situation; and means for evaluating the value of the information based on the emotion data and rarity contained in the generated information and automatically setting the price at which it is provided. This makes it possible to generate and provide information with accurate real-time market value based on information including emotions provided by the user.

[0156] A "user" refers to an individual who uses the system to photograph, provide, or purchase information.

[0157] "Image information" refers to still images or video data captured by a user using a digital device.

[0158] "Location data" refers to the geographical coordinate information of the user's current location, obtained by the user's device.

[0159] "Emotional data" refers to data that analyzes the user's emotional state based on captured image information.

[0160] A "generation device" refers to a mechanism that analyzes received data and generates information about a specific situation.

[0161] "Scarcity" refers to the concept that indicates the degree of uniqueness and value of generated information.

[0162] "Value assessment" refers to the process of determining the market price and usefulness of generated information.

[0163] "Automatic setting methods" refer to systems that use algorithms to automatically determine the price of information provision without human intervention.

[0164] "Information provision means" refers to a function within a system that allows other users to search for and purchase the generated information.

[0165] "Reward" refers to the monetary compensation received by a user who provides information.

[0166] "Electronic transaction methods" refer to systems that use electronic means to make payments.

[0167] This invention is a system that generates information about a specific situation based on image information, location data, and emotion data acquired by a user using a mobile device, and provides this information to other users.

[0168] The user first launches a dedicated application on their mobile device and takes images of the location. During this process, the device automatically acquires location data using its built-in GPS function and analyzes the user's emotions based on the image data using an emotion recognition engine. This data is then transmitted from the device to the server. In this case, the emotion recognition engine on the device is general emotion analysis software that uses machine learning algorithms to determine emotions.

[0169] The server uses a generative AI model to analyze the received data. The server processes this data using the generative AI model, generating specific situational information based on image and location data, and integrating sentiment data to evaluate the value of that information. The generated information is stored in a database, which other users can later access, search, and purchase through an application. In this process, the server automatically sets the price of the information based on its rarity and sentiment data.

[0170] For example, if a user visits a theme park and takes a picture of an attraction, this image is sent to the server along with its location data. Emotional data, such as the user's level of "fun" or "excitement," analyzed by the emotion engine, is also sent to the server. As a result, other users can learn not only about the crowd level of the place, but also the emotional reactions of other people there, and use this information to plan their visits.

[0171] An example of a prompt message could be: "Analyze the photos taken with my smartphone and my current emotional state, and generate a travel report." This prompt allows the server to provide detailed information, including the emotions the user felt.

[0172] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0173] Step 1:

[0174] The user launches a dedicated application on their mobile device and uses the camera function to take images of the site. The input is image information of the subject selected by the user, and an image file is generated as output. At this time, the user may have certain emotions, so this data will also be considered later by the system.

[0175] Step 2:

[0176] The device automatically acquires location data by measuring current geographical information using its built-in GPS module. The input is coordinate information obtained from the device's GPS sensor, and the output is location data at the time of shooting. This location data, along with image information, is used for subsequent analysis.

[0177] Step 3:

[0178] The device uses its built-in emotion recognition engine to analyze user emotion data from captured image information. The input is image information, and the output is data representing the user's emotion (e.g., happiness, excitement, surprise, etc.). This emotion data is sent to the server along with the image information and location data.

[0179] Step 4:

[0180] The terminal integrates acquired image information, location data, and sentiment data to form a data packet for transmission to the server. The input consists of image, location, and sentiment information stored within the terminal, while the output is an integrated data packet containing these. This packet is then transmitted to the server via the network.

[0181] Step 5:

[0182] The server analyzes the received data and processes the information using a generative AI model. The input is an integrated data packet sent from the terminal, and the output is the analyzed situational information. This process involves image recognition and sentiment analysis, and generates comprehensive information by integrating each piece of data.

[0183] Step 6:

[0184] The server evaluates the value of the generated information based on sentiment data and information scarcity, and automatically sets the price at which it is offered. The input is the generated contextual information and evaluation criteria, and the output is the selling price of the information. This clarifies the market value of the information.

[0185] Step 7:

[0186] The server stores the generated information in a database, making it searchable by other users. Input is the information to be stored, and output is the information entry in the database. This information is used when other users access and purchase it through the application.

[0187] (Application Example 2)

[0188] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0189] In today's world, users seek real-time, useful information, but traditional systems have failed to adequately provide information that reflects specific emotions and its actual value. Furthermore, challenges exist regarding fair pricing based on the value of the information and appropriate compensation for information providers.

[0190] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0191] In this invention, the server includes a device for receiving image data and location information captured by a user, a generating device for analyzing the user's emotions based on the image data and location information and generating information about a specific situation, and a device for automatically setting a price for providing the generated information based on its rarity and emotional information. This enables the provision of useful information that reflects the user's emotions, as well as fair pricing and compensation payments commensurate with its value.

[0192] A "user" is an entity that uses the system to capture image data and provide information.

[0193] "Image data" refers to visual information obtained from photos and videos taken by users.

[0194] "Location information" refers to data that indicates the geographical location of the user's photo.

[0195] "Emotions" refers to the result of an analysis engine evaluating the psychological state the user felt at the time of shooting.

[0196] A "generation device" is a device that analyzes image data and location information to create information that represents a specific situation.

[0197] "Scarcity" is a measure that evaluates the uniqueness and rarity of generated information.

[0198] A "device that automatically sets prices" is a device that automatically determines the selling price according to the value of the generated information.

[0199] An "information provision device" is a device that processes generated information so that users can search for and purchase it.

[0200] A "memory device" is a device that has the role of storing and managing generated information.

[0201] A "device for processing payment of rewards" is a device that provides rewards to information providers when information is purchased.

[0202] To realize this invention, a system is constructed that uses small portable devices such as smartphones and tablets, and a cloud server.

[0203] In this system, users take images and videos using the camera on their mobile device, and their location information is acquired using the device's GPS function. This acquired data is sent to a server via a mobile application. Furthermore, an emotion analysis engine installed in the device analyzes the captured images and videos and extracts the user's emotional state at that moment as digital data. This engine identifies emotions by analyzing facial expressions and scenes in images, for example, using AI functions from Google Cloud.

[0204] The server integrates received image data, location information, and sentiment data to generate information relevant to a specific situation. This generation process includes analyzing congestion levels and people's sentiment tendencies in real time and providing this data. Based on the scarcity of the generated information and the sentiment data, a pricing algorithm automatically calculates the selling price for that information.

[0205] By storing information in cloud storage, other users can search for and purchase the information through the application based on geographical conditions and keywords. When a purchase is completed, the server automatically pays the information provider (user) a reward via an electronic payment system.

[0206] As a concrete example, a user visits a popular tourist destination, takes photos, and captures the excitement they feel at that time. The app analyzes this data and provides information to other users indicating that the place is "highly satisfying."

[0207] An example of a prompt message would be, "Take a picture and tell us how you felt at the time. Which place did you like best?" In this way, it is possible to provide real-time content that leverages the user's experience.

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The user uses their device to take images and videos of tourist destinations and events. During this process, location information is automatically acquired using the device's GPS function. The input consists of image data from the user and location information acquired by the device. The output is a data package for transmission to the next process.

[0211] Step 2:

[0212] The device sends captured image data to its built-in emotion analysis engine to analyze the user's emotions. The analysis utilizes facial recognition and expression analysis within the image. The input is image data, and the output is emotion data reflecting the user's feelings.

[0213] Step 3:

[0214] The device integrates image data, location information, and sentiment data and sends it to the server. The input consists of various analyzed data, and the output is a request for integrated data to be sent to the server.

[0215] Step 4:

[0216] The server analyzes the received integrated data and generates specific situational information. For example, it aggregates data to infer the level of congestion in a tourist area or the overall sentiment. The input is integrated data, and the output is the analyzed situational information.

[0217] Step 5:

[0218] The server evaluates the rarity and sentiment data of the generated situational information and automatically determines the selling price of the information using a pricing algorithm. The input is the situational information and evaluation criteria, and the output is the set selling price.

[0219] Step 6:

[0220] The information is stored in the server's storage device and made available for users to search and purchase. The input is status information with a set sales price, and the output is the status of the information being provided to the user.

[0221] Step 7:

[0222] When a user purchases information, the server pays the information provider via electronic payment software. In this process, the purchase information is used as input, and the payment of the information provider is the output.

[0223] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0224] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0225] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0226] [Second Embodiment]

[0227] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0228] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0229] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0230] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0231] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0232] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0233] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0234] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0235] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0236] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0237] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0238] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0239] This invention relates to a system implemented via a smartphone or mobile device used by a user on a daily basis. The system begins with the user transmitting image data captured with the device and its location information to a server. The transmitted data is analyzed by a generation device on the server to generate specific situational information.

[0240] Image data captured by users will be obtained by combining a smartphone camera application with location services. Users can easily take images of a location using their device and automatically link the situation and geographical location to the image. For example, it could be used to inform other users about the congestion level at a tourist destination that a user has visited.

[0241] When the server receives data sent from a terminal, it first uses an image analysis algorithm based on the image data to identify individual elements (people, vehicles, sky conditions, etc.). This allows for estimations of congestion levels at tourist spots and theme parks, road traffic conditions, or weather conditions.

[0242] Location information, when combined with analysis results, is organized as local information for a specific region. This allows for the measurement of information scarcity and the evaluation of its value and demand to users. The server then sets the selling price of the information based on this scarcity data, but the selling price fluctuates depending on market demand and the availability of other similar information.

[0243] The generated information is stored in the server's storage and then made available for other users to access. Other users can use their devices to search for this information on the app and purchase it at an appropriate price. Once the purchase is complete, the server automatically processes the payment to the information provider via electronic payment. This system facilitates the smooth provision of data on a daily basis and its subsequent economic value creation.

[0244] This invention is particularly effective in realizing services targeting users residing in urban areas who are frequently on the move and have a high demand for real-time information, as well as in providing information related to the tourism industry.

[0245] The following describes the processing flow.

[0246] Step 1:

[0247] The user activates their smartphone and launches a dedicated application. Within the app, the user uses the camera function to take pictures of specific locations or objects. The device simultaneously uses its GPS function to collect location information at the time of photo capture.

[0248] Step 2:

[0249] The device uploads the captured image data and location information to the server. The data is transmitted securely using the HTTPS protocol. The device notifies the user that the transmission was successful.

[0250] Step 3:

[0251] The server receives image data and location information transmitted from the terminal. The server checks the data structure and format to ensure that there are no missing or incorrect data. Once validation is complete, the process moves on to analysis.

[0252] Step 4:

[0253] A generator on the server analyzes the image data and uses image recognition algorithms to identify elements in the photograph (e.g., people, cars, buildings, sky, etc.). The analysis results generate specific situational information (e.g., congestion level, traffic conditions, weather, etc.).

[0254] Step 5:

[0255] The server associates generated situational information with location information and evaluates the rarity of the information. Based on this rarity, the server automatically determines the selling price of the information. The price is set considering market demand and the availability of other similar information.

[0256] Step 6:

[0257] The server stores the generated information in a database and prepares it for other users to access and purchase later. This information can be searched using maps and keywords within the application.

[0258] Step 7:

[0259] The user (the user searching for information) uses the application to search for the information they need and select the information that interests them. The user reviews the details of the information and, if they wish to purchase it, proceeds with the purchase process.

[0260] Step 8:

[0261] Once the server confirms that the information has been purchased, it begins the process of paying the user who provided the information. The payment is made using the registered electronic payment method. This allows the information provider to receive economic compensation commensurate with the information they provided.

[0262] (Example 1)

[0263] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0264] In today's technological society, there is a demand for the efficient collection of highly accurate information in real time and the provision of that information as economic value. However, existing information provision systems have challenges such as limited user participation in information gathering and difficulty in setting prices based on the scarcity and market value of the information. Furthermore, a process for appropriately compensating information providers has not been established. This invention aims to solve these problems and provide a system that effectively connects the stages of information collection, analysis, and provision.

[0265] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0266] In this invention, the server includes means for processing image data and location information received via data equipment, an analysis device for analyzing the image data and location information and generating specific situational information, and means for automatically determining the price of the information based on the market evaluation of the generated situational information. This makes it possible to efficiently collect local information with user participation, quickly provide highly valuable information, and pay appropriate compensation to information providers.

[0267] A "data device" is an electronic device equipped with image capture and communication functions used by users to input information and transmit it to a server.

[0268] "Image data" refers to a digital representation of visual information captured by a user and transmitted through a data device.

[0269] "Location information" refers to geographical coordinate information acquired by data devices and transmitted along with image data.

[0270] "Means of processing" refers to the procedures or systems that allow the server to receive the received image data and location information and convert it into an analyzable format.

[0271] "Analysis" is data processing that uses algorithms based on image data and location information to identify specific environments or situations.

[0272] An "analysis device" is a part of a system that generates situational information from image data and location information using algorithms such as generative AI models.

[0273] "Situational information" refers to generated information that indicates the local conditions at a specific location, based on analyzed image data and location information.

[0274] "Market valuation" is the process of evaluating the value of generated situational information based on its scarcity and demand.

[0275] "Means for automatically determining the offering price" refers to a system function for setting the selling price of information based on market evaluation.

[0276] This invention is a system for efficiently collecting and utilizing information. Users take photographs of local conditions using a smartphone or mobile device, and the device's location information function acquires geographic coordinates during this process. The software used consists of a camera application and location services on the smartphone.

[0277] Image data and location information captured by the user are transmitted to the server via the device. Data transmission is securely performed using encrypted communication protocols (e.g., HTTPS).

[0278] The server analyzes the received image data using a generative AI model. This analysis process recognizes people, vehicles, and environmental elements (such as weather) within the image, and generates local situation information based on this recognition. Specific software used includes image analysis libraries and AI frameworks.

[0279] The generated status information is stored on the server, and pricing is automatically set according to market value. This pricing is dynamically adjusted by an algorithm that reflects market valuation.

[0280] Other users can access the server through an application on their device and purchase information they are interested in. When information is purchased, the purchase process is carried out through an electronic payment system, and once the purchase is complete, the server pays a reward to the information provider. This reward is processed through PayPal or bank APIs.

[0281] For example, if a user takes a photo of the crowd situation at a tourist spot and sends the information to the server, and that information is purchased by another user, the user who provided the information will be paid a reward.

[0282] Example prompt sentence: "Please provide the latest information for evaluating the current congestion situation and weather of the tourist destination Kiyomizu Temple."

[0283] In this way, the system realizes collecting real-world information in real time and maximizing its value.

[0284] The flow of the specific process in Example 1 will be described using FIG. 11.

[0285] Step 1:

[0286] The user uses the camera application of the smartphone to take a picture of the local area. At this time, the location information service is activated, and the latitude and longitude information is associated with the image data. The input is the image taken by the user and the location information by GPS, and the output is the dataset combining them. This dataset is transmitted from the terminal to the server.

[0287] Step 2:

[0288] The server receives the transmitted dataset. The processes performed here are storage and initial formatting of the image data and location information. Specifically, the image data is converted into an analyzable format, and the location information is processed into text format. Also, a data integrity check is performed to enhance the certainty of transmission.

[0289] Step 3:

[0290] The server executes image analysis using the generated AI model on the received dataset. It receives the image data as input and identifies the components and environmental information (such as people, vehicles, weather, etc.) in the image by the analysis algorithm. The output is the situation information including the identified information. As a result, specific situations such as the local congestion level and weather conditions are generated.

[0291] Step 4:

[0292] The server automatically sets prices for the generated situational information based on market valuations. Specifically, it stores the situational information in a database and applies an algorithm that determines the price by referencing similar historical data and market demand. The output of this process is the situational information with a sales price assigned to it.

[0293] Step 5:

[0294] Other users access the server using a smartphone app to search for and purchase situational information of interest. The input is the user's search query, and the server provides a list of information with prices as output. The purchase process then proceeds, and the transaction is completed using electronic payment methods.

[0295] Step 6:

[0296] Once a user completes their purchase of information, the server pays the information provider. Specifically, electronic payment is made via PayPal or a bank API based on the payment information. The inputs are purchase information and payment information, and the output is a notification that the payment has been completed. This allows information providers to receive compensation for the data they provide.

[0297] (Application Example 1)

[0298] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0299] In modern urban life, food delivery services are rapidly expanding, but delivery partners face unpredictable factors such as traffic congestion and weather changes, making fast and efficient deliveries difficult. Therefore, optimizing delivery routes and sharing real-time status information are essential.

[0300] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0301] In this invention, the server includes a device that receives image data and location information captured by a user, a device that analyzes the image data and location information to generate information regarding a specific situation, and a function that analyzes situation data on the delivery route and provides an optimized delivery route. As a result, it becomes possible for delivery partners to grasp the local traffic conditions and weather in real time and perform deliveries on a more efficient route.

[0302] The "user" refers to an individual or a corporation that uses information and services, and is the entity that captures image data and provides location information.

[0303] The "image data" refers to visual information captured by a user through a device, and is the basic data for analyzing a specific situation.

[0304] The "location information" refers to information indicating the geographical coordinates where the target image data was captured, and is used to grasp the situation in a specific area.

[0305] The "generation device" refers to a device or system that analyzes the received image data and location information to generate specific situation information.

[0306] The "rarity" refers to the criteria for evaluating the value and demand of the generated information based on factors such as the frequency of information provision and the existence of similar information.

[0307] The "information provision function" refers to the function of storing the generated information so that other users can search for and purchase it.

[0308] The "reward" refers to the consideration paid for the information sold by the information provider to other users, and has a monetary value or a value equivalent thereto.

[0309] The "traffic situation" refers to information regarding factors that affect movement, such as road congestion, traffic jams, and traffic volume.

[0310] "Delivery route optimization" refers to adjusting delivery routes to reach the destination in the shortest time or at the lowest cost.

[0311] This system is based on users collecting environmental information using smart devices and analyzing it on a server. Users efficiently acquire image data and location information using devices such as smartphones. Specifically, it utilizes the camera and location services (for example, Android's FusedLocationProviderClient or iOS's Core Location) built into the smartphone. When a user takes a picture at a designated location, the image data is automatically acquired and sent to the server along with the current location information.

[0312] The server receives transmitted image data and location information and processes this data using image analysis algorithms such as OpenCV. For example, it recognizes traffic conditions, congestion, and weather conditions from the image data and performs analysis in conjunction with the location information. Based on these analysis results, information is generated in real time, and prices are automatically set according to the rarity of the information.

[0313] The server then stores the generated information in cloud storage, allowing users to search for and purchase the information as needed through the application. This system automatically pays information providers electronically for the information they purchase.

[0314] As a further example, if a delivery partner records traffic congestion or weather changes during their delivery, the app can capture these scenes, analyze them in real time, and suggest the best route for other delivery partners. An example of a prompt message in this case would be, "Analyze the images showing traffic congestion and optimize your delivery route."

[0315] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0316] Step 1:

[0317] The user takes photos of their surroundings using a smart device. The input consists of image data and location information captured by the device's camera. The device uses location services to add the current geographic coordinates to the captured images. The output is a dataset combining the image data and location information.

[0318] Step 2:

[0319] The device sends the acquired image data and location information to the server. The input is a dataset captured by the user. This is securely transmitted to the server using protocols such as HTTPS. The output is the image data and location information received by the server.

[0320] Step 3:

[0321] The server performs image analysis using the received data. The input is image data received from the terminal. The server uses libraries such as OpenCV and TensorFlow to analyze features in the image and extract information such as traffic conditions, weather, and congestion. The output is situational information generated as a result of the analysis.

[0322] Step 4:

[0323] The server combines the situational information obtained through analysis with location information to assess the rarity of the information. The inputs are the situational information and location information generated in step 3. The server applies an algorithm to assess rarity and calculates the market value of the information. The output is the price at which the information is offered.

[0324] Step 5:

[0325] The server stores the generated information in a database, making it accessible to users. The input is information containing the value proposition calculated in step 4. The server registers this information in a cloud-based database. The output is the stored information, which can be searched and retrieved by other users.

[0326] Step 6:

[0327] When a user purchases information, the server processes the payment and pays the information provider. The inputs are the user's purchase request and payment information. The server uses an electronic payment service to execute the transaction and distribute the reward to the information provider. The outputs are a transaction completion notification and the payment of the reward.

[0328] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0329] This invention provides an information generation system incorporating emotion recognition technology, accessible via a mobile device typically used by the user. This system is characterized by its ability to analyze the user's emotions using an emotion engine, in addition to image data and location information captured by the user with a smartphone or similar device. As a result, the generated information can reflect the emotions the user felt at that moment.

[0330] The user first launches a dedicated application and uses its camera function to capture images of the site. At this time, the device automatically acquires location information and sends this information to the server. Furthermore, based on the captured images, the device analyzes the user's emotions via an emotion engine and sends the results to the server as additional data.

[0331] When the server receives this data, a generating device analyzes the image data and location information to generate information about a specific situation. For example, it can generate information that integrates not only the congestion level of a tourist spot but also the emotions of the people there (such as excitement, surprise, or boredom). This emotional information is integrated as an element that enhances the value of the situational information.

[0332] Furthermore, the server has an algorithm that evaluates the value of the generated information based on scarcity and sentiment data, and automatically sets the selling price. If the user's sentiment is positive, the information's value may increase, and as a result, the selling price of the information will be adjusted.

[0333] The generated information is stored in a database and later made accessible to other users. Users who need the information can search for it through the application based on geographical conditions and keywords, and then choose to purchase it. Once a purchase is completed, the server pays the user who provided the information through an electronic payment method.

[0334] As an example, when a user visits a theme park on a holiday and sends images of scenery and events taken at the park through the app, the emotion engine analyzes their level of enjoyment and satisfaction and adds this information. Other users can obtain this information through the app, allowing them to understand not only the detailed situation at the visited location but also the genuine emotions of the people there. By providing real-time information including emotion data in this way, information purchasers can make higher-quality decisions.

[0335] This invention is expected to be particularly useful in the tourism industry, event management, and other industries where real-time feedback is beneficial.

[0336] The following describes the processing flow.

[0337] Step 1:

[0338] The user launches a dedicated application on their smartphone and uses the photo-taking function to capture images of the site. Simultaneously, the device utilizes its GPS function to collect precise location information.

[0339] Step 2:

[0340] The device runs an emotion engine based on the captured image, analyzing the user's facial expressions and other image elements to estimate the user's emotions. The estimated emotion data is packaged together with the image data and location information.

[0341] Step 3:

[0342] The device sends the packaged data to the server. For security reasons, data transmission is performed via the HTTPS protocol. Users can confirm within the app that the data has been successfully uploaded.

[0343] Step 4:

[0344] The server analyzes the received image data, location information, and sentiment data. Based on this information, the generator produces specific situational information (for example, the level of crowding at a theme park and the level of enjoyment of visitors there).

[0345] Step 5:

[0346] The server combines generated situational information with sentiment data to assess its rarity and calculate its economic value. The selling price of the information is automatically set based on its rarity and the intensity of the sentiment.

[0347] Step 6:

[0348] The server stores the evaluated information in a database, making it accessible to other users through the app. This process ensures that the information is properly categorized and designed to be searchable.

[0349] Step 7:

[0350] Users who need information can search for it within the application and select the information that suits their purpose. After reviewing the details of the selected information, they can proceed with the purchase.

[0351] Step 8:

[0352] Once the purchase of information is complete, the server initiates the process of paying the user who provided the information. Payment is made using a pre-registered electronic payment method. This process ensures that the information provider receives payment for their services.

[0353] (Example 2)

[0354] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0355] In modern information delivery systems, there is a problem in that the information provided by users is merely a collection of data and fails to adequately reflect the atmosphere and emotional value of the moment. In particular, in tourist destinations and events, there is a demand for information that includes the atmosphere and emotions of the place in real time, but conventional systems have had difficulty achieving this. In addition, the pricing of the generated information often does not accurately reflect supply and demand, and the payment of appropriate compensation to users has not been smooth.

[0356] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0357] In this invention, the server includes means for receiving image information, location data, and emotion data captured by a user; a generation device for analyzing the image information, location data, and emotion data and generating comprehensive information about a specific situation; and means for evaluating the value of the information based on the emotion data and rarity contained in the generated information and automatically setting the price at which it is provided. This makes it possible to generate and provide information with accurate real-time market value based on information including emotions provided by the user.

[0358] A "user" refers to an individual who uses the system to photograph, provide, or purchase information.

[0359] "Image information" refers to still images or video data captured by a user using a digital device.

[0360] "Location data" refers to the geographical coordinate information of the user's current location, obtained by the user's device.

[0361] "Emotional data" refers to data that analyzes the user's emotional state based on captured image information.

[0362] A "generation device" refers to a mechanism that analyzes received data and generates information about a specific situation.

[0363] "Scarcity" refers to the concept that indicates the degree of uniqueness and value of generated information.

[0364] "Value assessment" refers to the process of determining the market price and usefulness of generated information.

[0365] "Automatic setting methods" refer to systems that use algorithms to automatically determine the price of information provision without human intervention.

[0366] "Information provision means" refers to a function within a system that allows other users to search for and purchase the generated information.

[0367] "Reward" refers to the monetary compensation received by a user who provides information.

[0368] "Electronic transaction methods" refer to systems that use electronic means to make payments.

[0369] This invention is a system that generates information about a specific situation based on image information, location data, and emotion data acquired by a user using a mobile device, and provides this information to other users.

[0370] The user first launches a dedicated application on their mobile device and takes images of the location. During this process, the device automatically acquires location data using its built-in GPS function and analyzes the user's emotions based on the image data using an emotion recognition engine. This data is then transmitted from the device to the server. In this case, the emotion recognition engine on the device is general emotion analysis software that uses machine learning algorithms to determine emotions.

[0371] The server uses a generative AI model to analyze the received data. The server processes this data using the generative AI model, generating specific situational information based on image and location data, and integrating sentiment data to evaluate the value of that information. The generated information is stored in a database, which other users can later access, search, and purchase through an application. In this process, the server automatically sets the price of the information based on its rarity and sentiment data.

[0372] For example, if a user visits a theme park and takes a picture of an attraction, this image is sent to the server along with its location data. Emotional data, such as the user's level of "fun" or "excitement," analyzed by the emotion engine, is also sent to the server. As a result, other users can learn not only about the crowd level of the place, but also the emotional reactions of other people there, and use this information to plan their visits.

[0373] An example of a prompt message could be: "Analyze the photos taken with my smartphone and my current emotional state, and generate a travel report." This prompt allows the server to provide detailed information, including the emotions the user felt.

[0374] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0375] Step 1:

[0376] The user launches a dedicated application on their mobile device and uses the camera function to take images of the site. The input is image information of the subject selected by the user, and an image file is generated as output. At this time, the user may have certain emotions, so this data will also be considered later by the system.

[0377] Step 2:

[0378] The device automatically acquires location data by measuring current geographical information using its built-in GPS module. The input is coordinate information obtained from the device's GPS sensor, and the output is location data at the time of shooting. This location data, along with image information, is used for subsequent analysis.

[0379] Step 3:

[0380] The device uses its built-in emotion recognition engine to analyze user emotion data from captured image information. The input is image information, and the output is data representing the user's emotion (e.g., happiness, excitement, surprise, etc.). This emotion data is sent to the server along with the image information and location data.

[0381] Step 4:

[0382] The terminal integrates acquired image information, location data, and sentiment data to form a data packet for transmission to the server. The input consists of image, location, and sentiment information stored within the terminal, while the output is an integrated data packet containing these. This packet is then transmitted to the server via the network.

[0383] Step 5:

[0384] The server analyzes the received data and processes the information using a generative AI model. The input is an integrated data packet sent from the terminal, and the output is the analyzed situational information. This process involves image recognition and sentiment analysis, and generates comprehensive information by integrating each piece of data.

[0385] Step 6:

[0386] The server evaluates the value of the generated information based on sentiment data and information scarcity, and automatically sets the price at which it is offered. The input is the generated contextual information and evaluation criteria, and the output is the selling price of the information. This clarifies the market value of the information.

[0387] Step 7:

[0388] The server stores the generated information in a database, making it searchable by other users. Input is the information to be stored, and output is the information entry in the database. This information is used when other users access and purchase it through the application.

[0389] (Application Example 2)

[0390] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0391] In today's world, users seek real-time, useful information, but traditional systems have failed to adequately provide information that reflects specific emotions and its actual value. Furthermore, challenges exist regarding fair pricing based on the value of the information and appropriate compensation for information providers.

[0392] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0393] In this invention, the server includes a device for receiving image data and location information captured by a user, a generating device for analyzing the user's emotions based on the image data and location information and generating information about a specific situation, and a device for automatically setting a price for providing the generated information based on its rarity and emotional information. This enables the provision of useful information that reflects the user's emotions, as well as fair pricing and compensation payments commensurate with its value.

[0394] A "user" is an entity that uses the system to capture image data and provide information.

[0395] "Image data" refers to visual information obtained from photos and videos taken by users.

[0396] "Location information" refers to data that indicates the geographical location of the user's photo.

[0397] "Emotions" refers to the result of an analysis engine evaluating the psychological state the user felt at the time of shooting.

[0398] A "generation device" is a device that analyzes image data and location information to create information that represents a specific situation.

[0399] "Scarcity" is a measure that evaluates the uniqueness and rarity of generated information.

[0400] A "device that automatically sets prices" is a device that automatically determines the selling price according to the value of the generated information.

[0401] An "information provision device" is a device that processes generated information so that users can search for and purchase it.

[0402] A "memory device" is a device that has the role of storing and managing generated information.

[0403] A "device for processing payment of rewards" is a device that provides rewards to information providers when information is purchased.

[0404] To realize this invention, a system is constructed that uses small portable devices such as smartphones and tablets, and a cloud server.

[0405] In this system, users take images and videos using the camera on their mobile device, and their location information is acquired using the device's GPS function. This acquired data is sent to a server via a mobile application. Furthermore, an emotion analysis engine installed in the device analyzes the captured images and videos and extracts the user's emotional state at that moment as digital data. This engine identifies emotions by analyzing facial expressions and scenes in the images, for example, using Google Cloud's AI functions.

[0406] The server integrates received image data, location information, and sentiment data to generate information relevant to a specific situation. This generation process includes analyzing congestion levels and people's sentiment tendencies in real time and providing this data. Based on the scarcity of the generated information and the sentiment data, a pricing algorithm automatically calculates the selling price for that information.

[0407] By storing information in cloud storage, other users can search for and purchase the information through the application based on geographical conditions and keywords. When a purchase is completed, the server automatically pays the information provider (user) a reward via an electronic payment system.

[0408] As a concrete example, a user visits a popular tourist destination, takes photos, and captures the excitement they feel at that time. The app analyzes this data and provides information to other users indicating that the place is "highly satisfying."

[0409] An example of a prompt message would be, "Take a picture and tell us how you felt at the time. Which place did you like best?" In this way, it is possible to provide real-time content that leverages the user's experience.

[0410] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0411] Step 1:

[0412] The user uses their device to take images and videos of tourist destinations and events. During this process, location information is automatically acquired using the device's GPS function. The input consists of image data from the user and location information acquired by the device. The output is a data package for transmission to the next process.

[0413] Step 2:

[0414] The device sends captured image data to its built-in emotion analysis engine to analyze the user's emotions. The analysis utilizes facial recognition and expression analysis within the image. The input is image data, and the output is emotion data reflecting the user's feelings.

[0415] Step 3:

[0416] The device integrates image data, location information, and sentiment data and sends it to the server. The input consists of various analyzed data, and the output is a request for integrated data to be sent to the server.

[0417] Step 4:

[0418] The server analyzes the received integrated data and generates specific situational information. For example, it aggregates data to infer the level of congestion in a tourist area or the overall sentiment. The input is integrated data, and the output is the analyzed situational information.

[0419] Step 5:

[0420] The server evaluates the rarity and sentiment data of the generated situational information and automatically determines the selling price of the information using a pricing algorithm. The input is the situational information and evaluation criteria, and the output is the set selling price.

[0421] Step 6:

[0422] The information is stored in the server's storage device and made available for users to search and purchase. The input is status information with a set sales price, and the output is the status of the information being provided to the user.

[0423] Step 7:

[0424] When a user purchases information, the server pays the information provider via electronic payment software. In this process, the purchase information is used as input, and the payment of the information provider is the output.

[0425] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0426] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0427] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0428] [Third Embodiment]

[0429] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0430] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0431] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0432] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0433] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0434] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0435] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0436] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0437] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0438] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0439] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0440] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0441] This invention relates to a system implemented via a smartphone or mobile device used by a user on a daily basis. The system begins with the user transmitting image data captured with the device and its location information to a server. The transmitted data is analyzed by a generation device on the server to generate specific situational information.

[0442] Image data captured by users will be obtained by combining a smartphone camera application with location services. Users can easily take images of a location using their device and automatically link the situation and geographical location to the image. For example, it could be used to inform other users about the congestion level at a tourist destination that a user has visited.

[0443] When the server receives data sent from a terminal, it first uses an image analysis algorithm based on the image data to identify individual elements (people, vehicles, sky conditions, etc.). This allows for estimations of congestion levels at tourist spots and theme parks, road traffic conditions, or weather conditions.

[0444] Location information, when combined with analysis results, is organized as local information for a specific region. This allows for the measurement of information scarcity and the evaluation of its value and demand to users. The server then sets the selling price of the information based on this scarcity data, but the selling price fluctuates depending on market demand and the availability of other similar information.

[0445] The generated information is stored in the server's storage and then made available for other users to access. Other users can use their devices to search for this information on the app and purchase it at an appropriate price. Once the purchase is complete, the server automatically processes the payment to the information provider via electronic payment. This system facilitates the smooth provision of data on a daily basis and its subsequent economic value creation.

[0446] This invention is particularly effective in realizing services targeting users residing in urban areas who are frequently on the move and have a high demand for real-time information, as well as in providing information related to the tourism industry.

[0447] The following describes the processing flow.

[0448] Step 1:

[0449] The user activates their smartphone and launches a dedicated application. Within the app, the user uses the camera function to take pictures of specific locations or objects. The device simultaneously uses its GPS function to collect location information at the time of photo capture.

[0450] Step 2:

[0451] The device uploads the captured image data and location information to the server. The data is transmitted securely using the HTTPS protocol. The device notifies the user that the transmission was successful.

[0452] Step 3:

[0453] The server receives image data and location information transmitted from the terminal. The server checks the data structure and format to ensure that there are no missing or incorrect data. Once validation is complete, the process moves on to analysis.

[0454] Step 4:

[0455] A generator on the server analyzes the image data and uses image recognition algorithms to identify elements in the photograph (e.g., people, cars, buildings, sky, etc.). The analysis results generate specific situational information (e.g., congestion level, traffic conditions, weather, etc.).

[0456] Step 5:

[0457] The server associates generated situational information with location information and evaluates the rarity of the information. Based on this rarity, the server automatically determines the selling price of the information. The price is set considering market demand and the availability of other similar information.

[0458] Step 6:

[0459] The server stores the generated information in a database and prepares it for other users to access and purchase later. This information can be searched using maps and keywords within the application.

[0460] Step 7:

[0461] The user (the user searching for information) uses the application to search for the information they need and select the information that interests them. The user reviews the details of the information and, if they wish to purchase it, proceeds with the purchase process.

[0462] Step 8:

[0463] Once the server confirms that the information has been purchased, it begins the process of paying the user who provided the information. The payment is made using the registered electronic payment method. This allows the information provider to receive economic compensation commensurate with the information they provided.

[0464] (Example 1)

[0465] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0466] In today's technological society, there is a demand for the efficient collection of highly accurate information in real time and the provision of that information as economic value. However, existing information provision systems have challenges such as limited user participation in information gathering and difficulty in setting prices based on the scarcity and market value of the information. Furthermore, a process for appropriately compensating information providers has not been established. This invention aims to solve these problems and provide a system that effectively connects the stages of information collection, analysis, and provision.

[0467] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0468] In this invention, the server includes means for processing image data and location information received via data equipment, an analysis device for analyzing the image data and location information and generating specific situational information, and means for automatically determining the price of the information based on the market evaluation of the generated situational information. This makes it possible to efficiently collect local information with user participation, quickly provide highly valuable information, and pay appropriate compensation to information providers.

[0469] A "data device" is an electronic device equipped with image capture and communication functions used by users to input information and transmit it to a server.

[0470] "Image data" refers to a digital representation of visual information captured by a user and transmitted through a data device.

[0471] "Location information" refers to geographical coordinate information acquired by data devices and transmitted along with image data.

[0472] "Means of processing" refers to the procedures or systems that allow the server to receive the received image data and location information and convert it into an analyzable format.

[0473] "Analysis" is data processing that uses algorithms based on image data and location information to identify specific environments or situations.

[0474] An "analysis device" is a part of a system that generates situational information from image data and location information using algorithms such as generative AI models.

[0475] "Situational information" refers to generated information that indicates the local conditions at a specific location, based on analyzed image data and location information.

[0476] "Market valuation" is the process of evaluating the value of generated situational information based on its scarcity and demand.

[0477] "Means for automatically determining the offering price" refers to a system function for setting the selling price of information based on market evaluation.

[0478] This invention is a system for efficiently collecting and utilizing information. Users take photographs of local conditions using a smartphone or mobile device, and the device's location information function acquires geographic coordinates during this process. The software used consists of a camera application and location services on the smartphone.

[0479] Image data and location information captured by the user are transmitted to the server via the device. Data transmission is securely performed using encrypted communication protocols (e.g., HTTPS).

[0480] The server analyzes the received image data using a generative AI model. This analysis process recognizes people, vehicles, and environmental elements (such as weather) within the image, and generates local situation information based on this recognition. Specific software used includes image analysis libraries and AI frameworks.

[0481] The generated status information is stored on the server, and pricing is automatically set according to market value. This pricing is dynamically adjusted by an algorithm that reflects market valuation.

[0482] Other users can access the server through an application on their device and purchase information they are interested in. When information is purchased, the purchase process is carried out through an electronic payment system, and once the purchase is complete, the server pays a reward to the information provider. This reward is processed through PayPal or bank APIs.

[0483] For example, if a user takes a photo of the crowd situation at a tourist spot and sends the information to the server, and that information is purchased by another user, the user who provided the information will be paid a reward.

[0484] Example prompt: "Please provide the latest information to assess the current crowd situation and weather at Kiyomizu-dera Temple, a popular tourist destination."

[0485] In this way, the system collects real-world information in real time and maximizes its value.

[0486] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0487] Step 1:

[0488] The user takes a picture of the location using their smartphone's camera application. At this time, location services are activated, and latitude and longitude information is linked to the image data. The input is the image taken by the user and the GPS location information, and the output is a dataset combining these two. This dataset is sent from the device to the server.

[0489] Step 2:

[0490] The server receives the transmitted dataset. The processing performed here involves storing and initially formatting the image data and location information. Specifically, the image data is converted into a parseable format, and the location information is processed into text format. In addition, data integrity checks are performed to enhance the reliability of the transmission.

[0491] Step 3:

[0492] The server performs image analysis on the received dataset using a generative AI model. It receives image data as input and uses an analysis algorithm to identify elements and environmental information (people, vehicles, weather, etc.) within the image. The output is situational information containing the identified information. This generates specific details about the location, such as congestion levels and weather conditions.

[0493] Step 4:

[0494] The server automatically sets prices for the generated situational information based on market valuations. Specifically, it stores the situational information in a database and applies an algorithm that determines the price by referencing similar historical data and market demand. The output of this process is the situational information with a sales price assigned to it.

[0495] Step 5:

[0496] Other users access the server using a smartphone app to search for and purchase situational information of interest. The input is the user's search query, and the server provides a list of information with prices as output. The purchase process then proceeds, and the transaction is completed using electronic payment methods.

[0497] Step 6:

[0498] Once a user completes their purchase of information, the server pays the information provider. Specifically, electronic payment is made via PayPal or a bank API based on the payment information. The inputs are purchase information and payment information, and the output is a notification that the payment has been completed. This allows information providers to receive compensation for the data they provide.

[0499] (Application Example 1)

[0500] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0501] In modern urban life, food delivery services are rapidly expanding, but delivery partners face unpredictable factors such as traffic congestion and weather changes, making fast and efficient deliveries difficult. Therefore, optimizing delivery routes and sharing real-time status information are essential.

[0502] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0503] In this invention, the server includes a device for receiving image data and location information captured by a user, a device for analyzing the image data and location information and generating information related to a specific situation, and a function for analyzing situation data along the delivery route and providing an optimized delivery route. This enables delivery partners to understand local traffic conditions and weather in real time and make deliveries using more efficient routes.

[0504] A "user" refers to an individual or legal entity that utilizes information or services, and is the entity that takes image data and provides location information.

[0505] "Image data" refers to visual information captured by a user through a device, and serves as basic data for analyzing specific situations.

[0506] "Location information" refers to information that indicates the geographical coordinates where the image data in question was taken, and is used to understand the situation in a specific area.

[0507] A "generation device" refers to a device or system that analyzes received image data and location information to generate specific situational information.

[0508] "Scarcity" is a criterion for evaluating the value and demand of information based on factors such as the frequency of information being provided and the existence of similar information.

[0509] The "information provision function" refers to a function that stores generated information and makes it available for other users to search for or purchase.

[0510] "Compensation" refers to the payment made by an information provider for information they sell to other users, and it is either monetary or of equivalent value.

[0511] "Traffic conditions" refers to information about factors that affect travel, such as road congestion, traffic jams, and traffic volume.

[0512] "Delivery route optimization" refers to adjusting delivery routes to reach the destination in the shortest time or at the lowest cost.

[0513] This system is based on users collecting environmental information using smart devices and analyzing it on a server. Users efficiently acquire image data and location information using devices such as smartphones. Specifically, it utilizes the camera and location services (for example, Android's FusedLocationProviderClient or iOS's Core Location) built into the smartphone. When a user takes a picture at a designated location, the image data is automatically acquired and sent to the server along with the current location information.

[0514] The server receives transmitted image data and location information and processes this data using image analysis algorithms such as OpenCV. For example, it recognizes traffic conditions, congestion, and weather conditions from the image data and performs analysis in conjunction with the location information. Based on these analysis results, information is generated in real time, and prices are automatically set according to the rarity of the information.

[0515] The server then stores the generated information in cloud storage, allowing users to search for and purchase the information as needed through the application. This system automatically pays information providers electronically for the information they purchase.

[0516] As a further example, if a delivery partner records traffic congestion or weather changes during their delivery, the app can capture these scenes, analyze them in real time, and suggest the best route for other delivery partners. An example of a prompt message in this case would be, "Analyze the images showing traffic congestion and optimize your delivery route."

[0517] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0518] Step 1:

[0519] The user takes photos of their surroundings using a smart device. The input consists of image data and location information captured by the device's camera. The device uses location services to add the current geographic coordinates to the captured images. The output is a dataset combining the image data and location information.

[0520] Step 2:

[0521] The device sends the acquired image data and location information to the server. The input is a dataset captured by the user. This is securely transmitted to the server using protocols such as HTTPS. The output is the image data and location information received by the server.

[0522] Step 3:

[0523] The server performs image analysis using the received data. The input is image data received from the terminal. The server uses libraries such as OpenCV and TensorFlow to analyze features in the image and extract information such as traffic conditions, weather, and congestion. The output is situational information generated as a result of the analysis.

[0524] Step 4:

[0525] The server combines the situational information obtained through analysis with location information to assess the rarity of the information. The inputs are the situational information and location information generated in step 3. The server applies an algorithm to assess rarity and calculates the market value of the information. The output is the price at which the information is offered.

[0526] Step 5:

[0527] The server stores the generated information in a database, making it accessible to users. The input is information containing the value proposition calculated in step 4. The server registers this information in a cloud-based database. The output is the stored information, which can be searched and retrieved by other users.

[0528] Step 6:

[0529] When a user purchases information, the server processes the payment and pays the information provider. The inputs are the user's purchase request and payment information. The server uses an electronic payment service to execute the transaction and distribute the reward to the information provider. The outputs are a transaction completion notification and the payment of the reward.

[0530] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0531] This invention provides an information generation system incorporating emotion recognition technology, accessible via a mobile device typically used by the user. This system is characterized by its ability to analyze the user's emotions using an emotion engine, in addition to image data and location information captured by the user with a smartphone or similar device. As a result, the generated information can reflect the emotions the user felt at that moment.

[0532] The user first launches a dedicated application and uses its camera function to capture images of the site. At this time, the device automatically acquires location information and sends this information to the server. Furthermore, based on the captured images, the device analyzes the user's emotions via an emotion engine and sends the results to the server as additional data.

[0533] When the server receives this data, a generating device analyzes the image data and location information to generate information about a specific situation. For example, it can generate information that integrates not only the congestion level of a tourist spot but also the emotions of the people there (such as excitement, surprise, or boredom). This emotional information is integrated as an element that enhances the value of the situational information.

[0534] Furthermore, the server has an algorithm that evaluates the value of the generated information based on scarcity and sentiment data, and automatically sets the selling price. If the user's sentiment is positive, the information's value may increase, and as a result, the selling price of the information will be adjusted.

[0535] The generated information is stored in a database and later made accessible to other users. Users who need the information can search for it through the application based on geographical conditions and keywords, and then choose to purchase it. Once a purchase is completed, the server pays the user who provided the information through an electronic payment method.

[0536] As an example, when a user visits a theme park on a holiday and sends images of scenery and events taken at the park through the app, the emotion engine analyzes their level of enjoyment and satisfaction and adds this information. Other users can obtain this information through the app, allowing them to understand not only the detailed situation at the visited location but also the genuine emotions of the people there. By providing real-time information including emotion data in this way, information purchasers can make higher-quality decisions.

[0537] This invention is expected to be particularly useful in the tourism industry, event management, and other industries where real-time feedback is beneficial.

[0538] The following describes the processing flow.

[0539] Step 1:

[0540] The user launches a dedicated application on their smartphone and uses the photo-taking function to capture images of the site. Simultaneously, the device utilizes its GPS function to collect precise location information.

[0541] Step 2:

[0542] The device runs an emotion engine based on the captured image, analyzing the user's facial expressions and other image elements to estimate the user's emotions. The estimated emotion data is packaged together with the image data and location information.

[0543] Step 3:

[0544] The device sends the packaged data to the server. For security reasons, data transmission is performed via the HTTPS protocol. Users can confirm within the app that the data has been successfully uploaded.

[0545] Step 4:

[0546] The server analyzes the received image data, location information, and sentiment data. Based on this information, the generator produces specific situational information (for example, the level of crowding at a theme park and the level of enjoyment of visitors there).

[0547] Step 5:

[0548] The server combines generated situational information with sentiment data to assess its rarity and calculate its economic value. The selling price of the information is automatically set based on its rarity and the intensity of the sentiment.

[0549] Step 6:

[0550] The server stores the evaluated information in a database, making it accessible to other users through the app. This process ensures that the information is properly categorized and designed to be searchable.

[0551] Step 7:

[0552] Users who need information can search for it within the application and select the information that suits their purpose. After reviewing the details of the selected information, they can proceed with the purchase.

[0553] Step 8:

[0554] Once the purchase of information is complete, the server initiates the process of paying the user who provided the information. Payment is made using a pre-registered electronic payment method. This process ensures that the information provider receives payment for their services.

[0555] (Example 2)

[0556] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0557] In modern information delivery systems, there is a problem in that the information provided by users is merely a collection of data and fails to adequately reflect the atmosphere and emotional value of the moment. In particular, in tourist destinations and events, there is a demand for information that includes the atmosphere and emotions of the place in real time, but conventional systems have had difficulty achieving this. In addition, the pricing of the generated information often does not accurately reflect supply and demand, and the payment of appropriate compensation to users has not been smooth.

[0558] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0559] In this invention, the server includes means for receiving image information, location data, and emotion data captured by a user; a generation device for analyzing the image information, location data, and emotion data and generating comprehensive information about a specific situation; and means for evaluating the value of the information based on the emotion data and rarity contained in the generated information and automatically setting the price at which it is provided. This makes it possible to generate and provide information with accurate real-time market value based on information including emotions provided by the user.

[0560] A "user" refers to an individual who uses the system to photograph, provide, or purchase information.

[0561] "Image information" refers to still images or video data captured by a user using a digital device.

[0562] "Location data" refers to the geographical coordinate information of the user's current location, obtained by the user's device.

[0563] "Emotional data" refers to data that analyzes the user's emotional state based on captured image information.

[0564] A "generation device" refers to a mechanism that analyzes received data and generates information about a specific situation.

[0565] "Scarcity" refers to the concept that indicates the degree of uniqueness and value of generated information.

[0566] "Value assessment" refers to the process of determining the market price and usefulness of generated information.

[0567] "Automatic setting methods" refer to systems that use algorithms to automatically determine the price of information provision without human intervention.

[0568] "Information provision means" refers to a function within a system that allows other users to search for and purchase the generated information.

[0569] "Reward" refers to the monetary compensation received by a user who provides information.

[0570] "Electronic transaction methods" refer to systems that use electronic means to make payments.

[0571] This invention is a system that generates information about a specific situation based on image information, location data, and emotion data acquired by a user using a mobile device, and provides this information to other users.

[0572] The user first launches a dedicated application on their mobile device and takes images of the location. During this process, the device automatically acquires location data using its built-in GPS function and analyzes the user's emotions based on the image data using an emotion recognition engine. This data is then transmitted from the device to the server. In this case, the emotion recognition engine on the device is general emotion analysis software that uses machine learning algorithms to determine emotions.

[0573] The server uses a generative AI model to analyze the received data. The server processes this data using the generative AI model, generating specific situational information based on image and location data, and integrating sentiment data to evaluate the value of that information. The generated information is stored in a database, which other users can later access, search, and purchase through an application. In this process, the server automatically sets the price of the information based on its rarity and sentiment data.

[0574] For example, if a user visits a theme park and takes a picture of an attraction, this image is sent to the server along with its location data. Emotional data, such as the user's level of "fun" or "excitement," analyzed by the emotion engine, is also sent to the server. As a result, other users can learn not only about the crowd level of the place, but also the emotional reactions of other people there, and use this information to plan their visits.

[0575] An example of a prompt message could be: "Analyze the photos taken with my smartphone and my current emotional state, and generate a travel report." This prompt allows the server to provide detailed information, including the emotions the user felt.

[0576] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0577] Step 1:

[0578] The user launches a dedicated application on their mobile device and uses the camera function to take images of the site. The input is image information of the subject selected by the user, and an image file is generated as output. At this time, the user may have certain emotions, so this data will also be considered later by the system.

[0579] Step 2:

[0580] The device automatically acquires location data by measuring current geographical information using its built-in GPS module. The input is coordinate information obtained from the device's GPS sensor, and the output is location data at the time of shooting. This location data, along with image information, is used for subsequent analysis.

[0581] Step 3:

[0582] The device uses its built-in emotion recognition engine to analyze user emotion data from captured image information. The input is image information, and the output is data representing the user's emotion (e.g., happiness, excitement, surprise, etc.). This emotion data is sent to the server along with the image information and location data.

[0583] Step 4:

[0584] The terminal integrates acquired image information, location data, and sentiment data to form a data packet for transmission to the server. The input consists of image, location, and sentiment information stored within the terminal, while the output is an integrated data packet containing these. This packet is then transmitted to the server via the network.

[0585] Step 5:

[0586] The server analyzes the received data and processes the information using a generative AI model. The input is an integrated data packet sent from the terminal, and the output is the analyzed situational information. This process involves image recognition and sentiment analysis, and generates comprehensive information by integrating each piece of data.

[0587] Step 6:

[0588] The server evaluates the value of the generated information based on sentiment data and information scarcity, and automatically sets the price at which it is offered. The input is the generated contextual information and evaluation criteria, and the output is the selling price of the information. This clarifies the market value of the information.

[0589] Step 7:

[0590] The server stores the generated information in a database, making it searchable by other users. Input is the information to be stored, and output is the information entry in the database. This information is used when other users access and purchase it through the application.

[0591] (Application Example 2)

[0592] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0593] In today's world, users seek real-time, useful information, but traditional systems have failed to adequately provide information that reflects specific emotions and its actual value. Furthermore, challenges exist regarding fair pricing based on the value of the information and appropriate compensation for information providers.

[0594] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0595] In this invention, the server includes a device for receiving image data and location information captured by a user, a generating device for analyzing the user's emotions based on the image data and location information and generating information about a specific situation, and a device for automatically setting a price for providing the generated information based on its rarity and emotional information. This enables the provision of useful information that reflects the user's emotions, as well as fair pricing and compensation payments commensurate with its value.

[0596] A "user" is an entity that uses the system to capture image data and provide information.

[0597] "Image data" refers to visual information obtained from photos and videos taken by users.

[0598] "Location information" refers to data that indicates the geographical location of the user's photo.

[0599] "Emotions" refers to the result of an analysis engine evaluating the psychological state the user felt at the time of shooting.

[0600] A "generation device" is a device that analyzes image data and location information to create information that represents a specific situation.

[0601] "Scarcity" is a measure that evaluates the uniqueness and rarity of generated information.

[0602] A "device that automatically sets prices" is a device that automatically determines the selling price according to the value of the generated information.

[0603] An "information provision device" is a device that processes generated information so that users can search for and purchase it.

[0604] A "memory device" is a device that has the role of storing and managing generated information.

[0605] A "device for processing payment of rewards" is a device that provides rewards to information providers when information is purchased.

[0606] To realize this invention, a system is constructed that uses small portable devices such as smartphones and tablets, and a cloud server.

[0607] In this system, users take images and videos using the camera on their mobile device, and their location information is acquired using the device's GPS function. This acquired data is sent to a server via a mobile application. Furthermore, an emotion analysis engine installed in the device analyzes the captured images and videos and extracts the user's emotional state at that moment as digital data. This engine identifies emotions by analyzing facial expressions and scenes in the images, for example, using Google Cloud's AI functions.

[0608] The server integrates received image data, location information, and sentiment data to generate information relevant to a specific situation. This generation process includes analyzing congestion levels and people's sentiment tendencies in real time and providing this data. Based on the scarcity of the generated information and the sentiment data, a pricing algorithm automatically calculates the selling price for that information.

[0609] By storing information in cloud storage, other users can search for and purchase the information through the application based on geographical conditions and keywords. When a purchase is completed, the server automatically pays the information provider (user) a reward via an electronic payment system.

[0610] As a concrete example, a user visits a popular tourist destination, takes photos, and captures the excitement they feel at that time. The app analyzes this data and provides information to other users indicating that the place is "highly satisfying."

[0611] An example of a prompt message would be, "Take a picture and tell us how you felt at the time. Which place did you like best?" In this way, it is possible to provide real-time content that leverages the user's experience.

[0612] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0613] Step 1:

[0614] The user uses their device to take images and videos of tourist destinations and events. During this process, location information is automatically acquired using the device's GPS function. The input consists of image data from the user and location information acquired by the device. The output is a data package for transmission to the next process.

[0615] Step 2:

[0616] The device sends captured image data to its built-in emotion analysis engine to analyze the user's emotions. The analysis utilizes facial recognition and expression analysis within the image. The input is image data, and the output is emotion data reflecting the user's feelings.

[0617] Step 3:

[0618] The device integrates image data, location information, and sentiment data and sends it to the server. The input consists of various analyzed data, and the output is a request for integrated data to be sent to the server.

[0619] Step 4:

[0620] The server analyzes the received integrated data and generates specific situational information. For example, it aggregates data to infer the level of congestion in a tourist area or the overall sentiment. The input is integrated data, and the output is the analyzed situational information.

[0621] Step 5:

[0622] The server evaluates the rarity and sentiment data of the generated situational information and automatically determines the selling price of the information using a pricing algorithm. The input is the situational information and evaluation criteria, and the output is the set selling price.

[0623] Step 6:

[0624] The information is stored in the server's storage device and made available for users to search and purchase. The input is status information with a set sales price, and the output is the status of the information being provided to the user.

[0625] Step 7:

[0626] When a user purchases information, the server pays the information provider via electronic payment software. In this process, the purchase information is used as input, and the payment of the information provider is the output.

[0627] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0628] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0629] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0630] [Fourth Embodiment]

[0631] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0632] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0633] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0634] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0635] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0636] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0637] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0638] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0639] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0640] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0641] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0642] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0643] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] This invention relates to a system implemented via a smartphone or mobile device used by a user on a daily basis. The system begins with the user transmitting image data captured with the device and its location information to a server. The transmitted data is analyzed by a generation device on the server to generate specific situational information.

[0645] Image data captured by users will be obtained by combining a smartphone camera application with location services. Users can easily take images of a location using their device and automatically link the situation and geographical location to the image. For example, it could be used to inform other users about the congestion level at a tourist destination that a user has visited.

[0646] When the server receives data sent from a terminal, it first uses an image analysis algorithm based on the image data to identify individual elements (people, vehicles, sky conditions, etc.). This allows for estimations of congestion levels at tourist spots and theme parks, road traffic conditions, or weather conditions.

[0647] Location information, when combined with analysis results, is organized as local information for a specific region. This allows for the measurement of information scarcity and the evaluation of its value and demand to users. The server then sets the selling price of the information based on this scarcity data, but the selling price fluctuates depending on market demand and the availability of other similar information.

[0648] The generated information is stored in the server's storage and then made available for other users to access. Other users can use their devices to search for this information on the app and purchase it at an appropriate price. Once the purchase is complete, the server automatically processes the payment to the information provider via electronic payment. This system facilitates the smooth provision of data on a daily basis and its subsequent economic value creation.

[0649] This invention is particularly effective in realizing services targeting users residing in urban areas who are frequently on the move and have a high demand for real-time information, as well as in providing information related to the tourism industry.

[0650] The following describes the processing flow.

[0651] Step 1:

[0652] The user activates their smartphone and launches a dedicated application. Within the app, the user uses the camera function to take pictures of specific locations or objects. The device simultaneously uses its GPS function to collect location information at the time of photo capture.

[0653] Step 2:

[0654] The device uploads the captured image data and location information to the server. The data is transmitted securely using the HTTPS protocol. The device notifies the user that the transmission was successful.

[0655] Step 3:

[0656] The server receives image data and location information transmitted from the terminal. The server checks the data structure and format to ensure that there are no missing or incorrect data. Once validation is complete, the process moves on to analysis.

[0657] Step 4:

[0658] A generator on the server analyzes the image data and uses image recognition algorithms to identify elements in the photograph (e.g., people, cars, buildings, sky, etc.). The analysis results generate specific situational information (e.g., congestion level, traffic conditions, weather, etc.).

[0659] Step 5:

[0660] The server associates generated situational information with location information and evaluates the rarity of the information. Based on this rarity, the server automatically determines the selling price of the information. The price is set considering market demand and the availability of other similar information.

[0661] Step 6:

[0662] The server stores the generated information in a database and prepares it for other users to access and purchase later. This information can be searched using maps and keywords within the application.

[0663] Step 7:

[0664] The user (the user searching for information) uses the application to search for the information they need and select the information that interests them. The user reviews the details of the information and, if they wish to purchase it, proceeds with the purchase process.

[0665] Step 8:

[0666] Once the server confirms that the information has been purchased, it begins the process of paying the user who provided the information. The payment is made using the registered electronic payment method. This allows the information provider to receive economic compensation commensurate with the information they provided.

[0667] (Example 1)

[0668] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0669] In today's technological society, there is a demand for the efficient collection of highly accurate information in real time and the provision of that information as economic value. However, existing information provision systems have challenges such as limited user participation in information gathering and difficulty in setting prices based on the scarcity and market value of the information. Furthermore, a process for appropriately compensating information providers has not been established. This invention aims to solve these problems and provide a system that effectively connects the stages of information collection, analysis, and provision.

[0670] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0671] In this invention, the server includes means for processing image data and location information received via data equipment, an analysis device for analyzing the image data and location information and generating specific situational information, and means for automatically determining the price of the information based on the market evaluation of the generated situational information. This makes it possible to efficiently collect local information with user participation, quickly provide highly valuable information, and pay appropriate compensation to information providers.

[0672] A "data device" is an electronic device equipped with image capture and communication functions used by users to input information and transmit it to a server.

[0673] "Image data" refers to a digital representation of visual information captured by a user and transmitted through a data device.

[0674] "Location information" refers to geographical coordinate information acquired by data devices and transmitted along with image data.

[0675] "Means of processing" refers to the procedures or systems that allow the server to receive the received image data and location information and convert it into an analyzable format.

[0676] "Analysis" is data processing that uses algorithms based on image data and location information to identify specific environments or situations.

[0677] An "analysis device" is a part of a system that generates situational information from image data and location information using algorithms such as generative AI models.

[0678] "Situational information" refers to generated information that indicates the local conditions at a specific location, based on analyzed image data and location information.

[0679] "Market valuation" is the process of evaluating the value of generated situational information based on its scarcity and demand.

[0680] "Means for automatically determining the offering price" refers to a system function for setting the selling price of information based on market evaluation.

[0681] This invention is a system for efficiently collecting and utilizing information. Users take photographs of local conditions using a smartphone or mobile device, and the device's location information function acquires geographic coordinates during this process. The software used consists of a camera application and location services on the smartphone.

[0682] Image data and location information captured by the user are transmitted to the server via the device. Data transmission is securely performed using encrypted communication protocols (e.g., HTTPS).

[0683] The server analyzes the received image data using a generative AI model. This analysis process recognizes people, vehicles, and environmental elements (such as weather) within the image, and generates local situation information based on this recognition. Specific software used includes image analysis libraries and AI frameworks.

[0684] The generated status information is stored on the server, and pricing is automatically set according to market value. This pricing is dynamically adjusted by an algorithm that reflects market valuation.

[0685] Other users can access the server through an application on their device and purchase information they are interested in. When information is purchased, the purchase process is carried out through an electronic payment system, and once the purchase is complete, the server pays a reward to the information provider. This reward is processed through PayPal or bank APIs.

[0686] For example, if a user takes a photo of the crowd situation at a tourist spot and sends the information to the server, and that information is purchased by another user, the user who provided the information will be paid a reward.

[0687] Example prompt: "Please provide the latest information to assess the current crowd situation and weather at Kiyomizu-dera Temple, a popular tourist destination."

[0688] In this way, the system collects real-world information in real time and maximizes its value.

[0689] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0690] Step 1:

[0691] The user takes a picture of the location using their smartphone's camera application. At this time, location services are activated, and latitude and longitude information is linked to the image data. The input is the image taken by the user and the GPS location information, and the output is a dataset combining these two. This dataset is sent from the device to the server.

[0692] Step 2:

[0693] The server receives the transmitted dataset. The processing performed here involves storing and initially formatting the image data and location information. Specifically, the image data is converted into a parseable format, and the location information is processed into text format. In addition, data integrity checks are performed to enhance the reliability of the transmission.

[0694] Step 3:

[0695] The server performs image analysis on the received dataset using a generative AI model. It receives image data as input and uses an analysis algorithm to identify elements and environmental information (people, vehicles, weather, etc.) within the image. The output is situational information containing the identified information. This generates specific details about the location, such as congestion levels and weather conditions.

[0696] Step 4:

[0697] The server automatically sets prices for the generated situational information based on market valuations. Specifically, it stores the situational information in a database and applies an algorithm that determines the price by referencing similar historical data and market demand. The output of this process is the situational information with a sales price assigned to it.

[0698] Step 5:

[0699] Other users access the server using a smartphone app to search for and purchase situational information of interest. The input is the user's search query, and the server provides a list of information with prices as output. The purchase process then proceeds, and the transaction is completed using electronic payment methods.

[0700] Step 6:

[0701] Once a user completes their purchase of information, the server pays the information provider. Specifically, electronic payment is made via PayPal or a bank API based on the payment information. The inputs are purchase information and payment information, and the output is a notification that the payment has been completed. This allows information providers to receive compensation for the data they provide.

[0702] (Application Example 1)

[0703] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0704] In modern urban life, food delivery services are rapidly expanding, but delivery partners face unpredictable factors such as traffic congestion and weather changes, making fast and efficient deliveries difficult. Therefore, optimizing delivery routes and sharing real-time status information are essential.

[0705] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0706] In this invention, the server includes a device for receiving image data and location information captured by a user, a device for analyzing the image data and location information and generating information related to a specific situation, and a function for analyzing situation data along the delivery route and providing an optimized delivery route. This enables delivery partners to understand local traffic conditions and weather in real time and make deliveries using more efficient routes.

[0707] A "user" refers to an individual or legal entity that utilizes information or services, and is the entity that takes image data and provides location information.

[0708] "Image data" refers to visual information captured by a user through a device, and serves as basic data for analyzing specific situations.

[0709] "Location information" refers to information that indicates the geographical coordinates where the image data in question was taken, and is used to understand the situation in a specific area.

[0710] A "generation device" refers to a device or system that analyzes received image data and location information to generate specific situational information.

[0711] "Scarcity" is a criterion for evaluating the value and demand of information based on factors such as the frequency of information being provided and the existence of similar information.

[0712] The "information provision function" refers to a function that stores generated information and makes it available for other users to search for or purchase.

[0713] "Compensation" refers to the payment made by an information provider for information they sell to other users, and it is either monetary or of equivalent value.

[0714] "Traffic conditions" refers to information about factors that affect travel, such as road congestion, traffic jams, and traffic volume.

[0715] "Delivery route optimization" refers to adjusting delivery routes to reach the destination in the shortest time or at the lowest cost.

[0716] This system is based on users collecting environmental information using smart devices and analyzing it on a server. Users efficiently acquire image data and location information using devices such as smartphones. Specifically, it utilizes the camera and location services (for example, Android's FusedLocationProviderClient or iOS's Core Location) built into the smartphone. When a user takes a picture at a designated location, the image data is automatically acquired and sent to the server along with the current location information.

[0717] The server receives transmitted image data and location information and processes this data using image analysis algorithms such as OpenCV. For example, it recognizes traffic conditions, congestion, and weather conditions from the image data and performs analysis in conjunction with the location information. Based on these analysis results, information is generated in real time, and prices are automatically set according to the rarity of the information.

[0718] The server then stores the generated information in cloud storage, allowing users to search for and purchase the information as needed through the application. This system automatically pays information providers electronically for the information they purchase.

[0719] As a further example, if a delivery partner records traffic congestion or weather changes during their delivery, the app can capture these scenes, analyze them in real time, and suggest the best route for other delivery partners. An example of a prompt message in this case would be, "Analyze the images showing traffic congestion and optimize your delivery route."

[0720] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0721] Step 1:

[0722] The user takes photos of their surroundings using a smart device. The input consists of image data and location information captured by the device's camera. The device uses location services to add the current geographic coordinates to the captured images. The output is a dataset combining the image data and location information.

[0723] Step 2:

[0724] The device sends the acquired image data and location information to the server. The input is a dataset captured by the user. This is securely transmitted to the server using protocols such as HTTPS. The output is the image data and location information received by the server.

[0725] Step 3:

[0726] The server performs image analysis using the received data. The input is image data received from the terminal. The server uses libraries such as OpenCV and TensorFlow to analyze features in the image and extract information such as traffic conditions, weather, and congestion. The output is situational information generated as a result of the analysis.

[0727] Step 4:

[0728] The server combines the situational information obtained through analysis with location information to assess the rarity of the information. The inputs are the situational information and location information generated in step 3. The server applies an algorithm to assess rarity and calculates the market value of the information. The output is the price at which the information is offered.

[0729] Step 5:

[0730] The server stores the generated information in a database, making it accessible to users. The input is information containing the value proposition calculated in step 4. The server registers this information in a cloud-based database. The output is the stored information, which can be searched and retrieved by other users.

[0731] Step 6:

[0732] When a user purchases information, the server processes the payment and pays the information provider. The inputs are the user's purchase request and payment information. The server uses an electronic payment service to execute the transaction and distribute the reward to the information provider. The outputs are a transaction completion notification and the payment of the reward.

[0733] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0734] This invention provides an information generation system incorporating emotion recognition technology, accessible via a mobile device typically used by the user. This system is characterized by its ability to analyze the user's emotions using an emotion engine, in addition to image data and location information captured by the user with a smartphone or similar device. As a result, the generated information can reflect the emotions the user felt at that moment.

[0735] The user first launches a dedicated application and uses its camera function to capture images of the site. At this time, the device automatically acquires location information and sends this information to the server. Furthermore, based on the captured images, the device analyzes the user's emotions via an emotion engine and sends the results to the server as additional data.

[0736] When the server receives this data, a generating device analyzes the image data and location information to generate information about a specific situation. For example, it can generate information that integrates not only the congestion level of a tourist spot but also the emotions of the people there (such as excitement, surprise, or boredom). This emotional information is integrated as an element that enhances the value of the situational information.

[0737] Furthermore, the server has an algorithm that evaluates the value of the generated information based on scarcity and sentiment data, and automatically sets the selling price. If the user's sentiment is positive, the information's value may increase, and as a result, the selling price of the information will be adjusted.

[0738] The generated information is stored in a database and later made accessible to other users. Users who need the information can search for it through the application based on geographical conditions and keywords, and then choose to purchase it. Once a purchase is completed, the server pays the user who provided the information through an electronic payment method.

[0739] As an example, when a user visits a theme park on a holiday and sends images of scenery and events taken at the park through the app, the emotion engine analyzes their level of enjoyment and satisfaction and adds this information. Other users can obtain this information through the app, allowing them to understand not only the detailed situation at the visited location but also the genuine emotions of the people there. By providing real-time information including emotion data in this way, information purchasers can make higher-quality decisions.

[0740] This invention is expected to be particularly useful in the tourism industry, event management, and other industries where real-time feedback is beneficial.

[0741] The following describes the processing flow.

[0742] Step 1:

[0743] The user launches a dedicated application on their smartphone and uses the photo-taking function to capture images of the site. Simultaneously, the device utilizes its GPS function to collect precise location information.

[0744] Step 2:

[0745] The device runs an emotion engine based on the captured image, analyzing the user's facial expressions and other image elements to estimate the user's emotions. The estimated emotion data is packaged together with the image data and location information.

[0746] Step 3:

[0747] The device sends the packaged data to the server. For security reasons, data transmission is performed via the HTTPS protocol. Users can confirm within the app that the data has been successfully uploaded.

[0748] Step 4:

[0749] The server analyzes the received image data, location information, and sentiment data. Based on this information, the generator produces specific situational information (for example, the level of crowding at a theme park and the level of enjoyment of visitors there).

[0750] Step 5:

[0751] The server combines generated situational information with sentiment data to assess its rarity and calculate its economic value. The selling price of the information is automatically set based on its rarity and the intensity of the sentiment.

[0752] Step 6:

[0753] The server stores the evaluated information in a database, making it accessible to other users through the app. This process ensures that the information is properly categorized and designed to be searchable.

[0754] Step 7:

[0755] Users who need information can search for it within the application and select the information that suits their purpose. After reviewing the details of the selected information, they can proceed with the purchase.

[0756] Step 8:

[0757] Once the purchase of information is complete, the server initiates the process of paying the user who provided the information. Payment is made using a pre-registered electronic payment method. This process ensures that the information provider receives payment for their services.

[0758] (Example 2)

[0759] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0760] In modern information delivery systems, there is a problem in that the information provided by users is merely a collection of data and fails to adequately reflect the atmosphere and emotional value of the moment. In particular, in tourist destinations and events, there is a demand for information that includes the atmosphere and emotions of the place in real time, but conventional systems have had difficulty achieving this. In addition, the pricing of the generated information often does not accurately reflect supply and demand, and the payment of appropriate compensation to users has not been smooth.

[0761] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0762] In this invention, the server includes means for receiving image information, location data, and emotion data captured by a user; a generation device for analyzing the image information, location data, and emotion data and generating comprehensive information about a specific situation; and means for evaluating the value of the information based on the emotion data and rarity contained in the generated information and automatically setting the price at which it is provided. This makes it possible to generate and provide information with accurate real-time market value based on information including emotions provided by the user.

[0763] A "user" refers to an individual who uses the system to photograph, provide, or purchase information.

[0764] "Image information" refers to still images or video data captured by a user using a digital device.

[0765] "Location data" refers to the geographical coordinate information of the user's current location, obtained by the user's device.

[0766] "Emotional data" refers to data that analyzes the user's emotional state based on captured image information.

[0767] A "generation device" refers to a mechanism that analyzes received data and generates information about a specific situation.

[0768] "Scarcity" refers to the concept that indicates the degree of uniqueness and value of generated information.

[0769] "Value assessment" refers to the process of determining the market price and usefulness of generated information.

[0770] "Automatic setting methods" refer to systems that use algorithms to automatically determine the price of information provision without human intervention.

[0771] "Information provision means" refers to a function within a system that allows other users to search for and purchase the generated information.

[0772] "Reward" refers to the monetary compensation received by a user who provides information.

[0773] "Electronic transaction methods" refer to systems that use electronic means to make payments.

[0774] This invention is a system that generates information about a specific situation based on image information, location data, and emotion data acquired by a user using a mobile device, and provides this information to other users.

[0775] The user first launches a dedicated application on their mobile device and takes images of the location. During this process, the device automatically acquires location data using its built-in GPS function and analyzes the user's emotions based on the image data using an emotion recognition engine. This data is then transmitted from the device to the server. In this case, the emotion recognition engine on the device is general emotion analysis software that uses machine learning algorithms to determine emotions.

[0776] The server uses a generative AI model to analyze the received data. The server processes this data using the generative AI model, generating specific situational information based on image and location data, and integrating sentiment data to evaluate the value of that information. The generated information is stored in a database, which other users can later access, search, and purchase through an application. In this process, the server automatically sets the price of the information based on its rarity and sentiment data.

[0777] For example, if a user visits a theme park and takes a picture of an attraction, this image is sent to the server along with its location data. Emotional data, such as the user's level of "fun" or "excitement," analyzed by the emotion engine, is also sent to the server. As a result, other users can learn not only about the crowd level of the place, but also the emotional reactions of other people there, and use this information to plan their visits.

[0778] An example of a prompt message could be: "Analyze the photos taken with my smartphone and my current emotional state, and generate a travel report." This prompt allows the server to provide detailed information, including the emotions the user felt.

[0779] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0780] Step 1:

[0781] The user launches a dedicated application on their mobile device and uses the camera function to take images of the site. The input is image information of the subject selected by the user, and an image file is generated as output. At this time, the user may have certain emotions, so this data will also be considered later by the system.

[0782] Step 2:

[0783] The device automatically acquires location data by measuring current geographical information using its built-in GPS module. The input is coordinate information obtained from the device's GPS sensor, and the output is location data at the time of shooting. This location data, along with image information, is used for subsequent analysis.

[0784] Step 3:

[0785] The device uses its built-in emotion recognition engine to analyze user emotion data from captured image information. The input is image information, and the output is data representing the user's emotion (e.g., happiness, excitement, surprise, etc.). This emotion data is sent to the server along with the image information and location data.

[0786] Step 4:

[0787] The terminal integrates acquired image information, location data, and sentiment data to form a data packet for transmission to the server. The input consists of image, location, and sentiment information stored within the terminal, while the output is an integrated data packet containing these. This packet is then transmitted to the server via the network.

[0788] Step 5:

[0789] The server analyzes the received data and processes the information using a generative AI model. The input is an integrated data packet sent from the terminal, and the output is the analyzed situational information. This process involves image recognition and sentiment analysis, and generates comprehensive information by integrating each piece of data.

[0790] Step 6:

[0791] The server evaluates the value of the generated information based on sentiment data and information scarcity, and automatically sets the price at which it is offered. The input is the generated contextual information and evaluation criteria, and the output is the selling price of the information. This clarifies the market value of the information.

[0792] Step 7:

[0793] The server stores the generated information in a database, making it searchable by other users. Input is the information to be stored, and output is the information entry in the database. This information is used when other users access and purchase it through the application.

[0794] (Application Example 2)

[0795] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0796] In today's world, users seek real-time, useful information, but traditional systems have failed to adequately provide information that reflects specific emotions and its actual value. Furthermore, challenges exist regarding fair pricing based on the value of the information and appropriate compensation for information providers.

[0797] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0798] In this invention, the server includes a device for receiving image data and location information captured by a user, a generating device for analyzing the user's emotions based on the image data and location information and generating information about a specific situation, and a device for automatically setting a price for providing the generated information based on its rarity and emotional information. This enables the provision of useful information that reflects the user's emotions, as well as fair pricing and compensation payments commensurate with its value.

[0799] A "user" is an entity that uses the system to capture image data and provide information.

[0800] "Image data" refers to visual information obtained from photos and videos taken by users.

[0801] "Location information" refers to data that indicates the geographical location of the user's photo.

[0802] "Emotions" refers to the result of an analysis engine evaluating the psychological state the user felt at the time of shooting.

[0803] A "generation device" is a device that analyzes image data and location information to create information that represents a specific situation.

[0804] "Scarcity" is a measure that evaluates the uniqueness and rarity of generated information.

[0805] A "device that automatically sets prices" is a device that automatically determines the selling price according to the value of the generated information.

[0806] An "information provision device" is a device that processes generated information so that users can search for and purchase it.

[0807] A "memory device" is a device that has the role of storing and managing generated information.

[0808] A "device for processing payment of rewards" is a device that provides rewards to information providers when information is purchased.

[0809] To realize this invention, a system is constructed that uses small portable devices such as smartphones and tablets, and a cloud server.

[0810] In this system, users take images and videos using the camera on their mobile device, and their location information is acquired using the device's GPS function. This acquired data is sent to a server via a mobile application. Furthermore, an emotion analysis engine installed in the device analyzes the captured images and videos and extracts the user's emotional state at that moment as digital data. This engine identifies emotions by analyzing facial expressions and scenes in the images, for example, using Google Cloud's AI functions.

[0811] The server integrates received image data, location information, and sentiment data to generate information relevant to a specific situation. This generation process includes analyzing congestion levels and people's sentiment tendencies in real time and providing this data. Based on the scarcity of the generated information and the sentiment data, a pricing algorithm automatically calculates the selling price for that information.

[0812] By storing information in cloud storage, other users can search for and purchase the information through the application based on geographical conditions and keywords. When a purchase is completed, the server automatically pays the information provider (user) a reward via an electronic payment system.

[0813] As a concrete example, a user visits a popular tourist destination, takes photos, and captures the excitement they feel at that time. The app analyzes this data and provides information to other users indicating that the place is "highly satisfying."

[0814] An example of a prompt message would be, "Take a picture and tell us how you felt at the time. Which place did you like best?" In this way, it is possible to provide real-time content that leverages the user's experience.

[0815] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0816] Step 1:

[0817] The user uses their device to take images and videos of tourist destinations and events. During this process, location information is automatically acquired using the device's GPS function. The input consists of image data from the user and location information acquired by the device. The output is a data package for transmission to the next process.

[0818] Step 2:

[0819] The device sends captured image data to its built-in emotion analysis engine to analyze the user's emotions. The analysis utilizes facial recognition and expression analysis within the image. The input is image data, and the output is emotion data reflecting the user's feelings.

[0820] Step 3:

[0821] The device integrates image data, location information, and sentiment data and sends it to the server. The input consists of various analyzed data, and the output is a request for integrated data to be sent to the server.

[0822] Step 4:

[0823] The server analyzes the received integrated data and generates specific situational information. For example, it aggregates data to infer the level of congestion in a tourist area or the overall sentiment. The input is integrated data, and the output is the analyzed situational information.

[0824] Step 5:

[0825] The server evaluates the rarity and sentiment data of the generated situational information and automatically determines the selling price of the information using a pricing algorithm. The input is the situational information and evaluation criteria, and the output is the set selling price.

[0826] Step 6:

[0827] The information is stored in the server's storage device and made available for users to search and purchase. The input is status information with a set sales price, and the output is the status of the information being provided to the user.

[0828] Step 7:

[0829] When a user purchases information, the server pays the information provider via electronic payment software. In this process, the purchase information is used as input, and the payment of the information provider is the output.

[0830] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0831] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0832] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0833] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0834] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0835] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0836] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0837] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0838] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0839] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0840] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0841] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0842] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0843] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0844] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0845] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0846] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0847] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0848] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0849] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0850] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0851] The following is further disclosed regarding the embodiments described above.

[0852] (Claim 1)

[0853] Means for receiving image data and location information captured by the user,

[0854] A generating device that analyzes the aforementioned image data and location information to generate information about a specific situation,

[0855] A means for automatically setting the price for providing the generated information based on its scarcity,

[0856] Information provision means that stores the aforementioned information in a storage device and allows the user to search for and purchase it,

[0857] A means for processing payment to the information provider when the aforementioned information is purchased,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, wherein the generating device comprises means for analyzing traffic conditions, congestion conditions, and weather information based on image data.

[0861] (Claim 3)

[0862] The system according to claim 1, wherein the payment processing of the aforementioned remuneration is carried out using an electronic payment method.

[0863] "Example 1"

[0864] (Claim 1)

[0865] Means for processing image data and location information received via data equipment,

[0866] An analysis device that analyzes the aforementioned image data and location information and generates specific situational information,

[0867] A means for automatically determining the price of information based on the market evaluation of the generated situational information,

[0868] A supply means that stores the generated information in a data storage device and allows users to search and purchase the information,

[0869] A means of processing payment to contributors when the information purchase is completed,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, wherein the analysis device comprises means for analyzing traffic conditions, congestion conditions, and weather information based on image data.

[0873] (Claim 3)

[0874] The system according to claim 1, wherein the payment processing of the aforementioned remuneration is carried out using an electronic payment method.

[0875] "Application Example 1"

[0876] (Claim 1)

[0877] A device that receives image data and location information captured by a user,

[0878] A device that analyzes the aforementioned image data and location information to generate information about a specific situation,

[0879] A device that automatically sets the price for providing the generated information based on its scarcity,

[0880] The aforementioned information is stored in a memory function, and an information provision function is provided that allows users to search for and purchase the information.

[0881] A device that processes payment of a reward to the information provider when the aforementioned information is purchased,

[0882] A system that includes a function to analyze status data along the delivery route and provide an optimized delivery route.

[0883] (Claim 2)

[0884] The system according to claim 1, wherein the generating device comprises means for analyzing traffic conditions, congestion conditions, and weather information based on image data and optimizing the delivery route.

[0885] (Claim 3)

[0886] The system according to claim 1, wherein the payment processing of the aforementioned remuneration is carried out using an electronic payment method.

[0887] "Example 2 of combining an emotion engine"

[0888] (Claim 1)

[0889] A means for receiving image information, location data, and emotion data captured by the user,

[0890] A generating device that analyzes the aforementioned image information, location data, and emotion data to generate comprehensive information about a specific situation,

[0891] A means for evaluating the value of the information based on the emotional data and scarcity contained in the generated information, and for automatically setting the price at which the information is offered.

[0892] Information provision means that stores the aforementioned information in a storage device and makes it searchable and available for purchase by other users,

[0893] A means for processing payment of a reward to the provider using electronic transaction methods when the aforementioned information is purchased,

[0894] A system that includes this.

[0895] (Claim 2)

[0896] The system according to claim 1, wherein the generating device comprises means for analyzing the state of a place, the level of congestion, or the emotional state based on image information.

[0897] (Claim 3)

[0898] The system according to claim 1, wherein the payment processing of the aforementioned remuneration is performed through electronic transaction means.

[0899] "Application example 2 when combining with an emotional engine"

[0900] (Claim 1)

[0901] A device that receives image data and location information captured by a user,

[0902] A generating device that analyzes the user's emotions based on the aforementioned image data and location information and generates information related to a specific situation,

[0903] A device that automatically sets the price for providing the generated information based on its scarcity and emotional information,

[0904] An information provision device that stores the aforementioned information in a storage device and allows users to search for and purchase it,

[0905] A device that processes payment of a reward to the information provider when the aforementioned information is purchased,

[0906] A system that includes this.

[0907] (Claim 2)

[0908] The system according to claim 1, wherein the generating device comprises a device for analyzing traffic conditions, congestion conditions, weather information, and emotional states based on image data.

[0909] (Claim 3)

[0910] The system according to claim 1, wherein the payment processing of the aforementioned remuneration is performed using an electronic payment device. [Explanation of Symbols]

[0911] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for receiving image data and location information captured by the user, A generating device that analyzes the aforementioned image data and location information to generate information about a specific situation, A means for automatically setting the price for providing the generated information based on its scarcity, Information provision means that stores the aforementioned information in a storage device and allows the user to search for and purchase it, A means for processing payment to the information provider when the aforementioned information is purchased, A system that includes this.

2. The system according to claim 1, wherein the generating device comprises means for analyzing traffic conditions, congestion conditions, and weather information based on image data.

3. The system according to claim 1, wherein the payment processing of the aforementioned remuneration is carried out using an electronic payment method.

Images