System

Abstract

A system is provided.SOLUTION: A system including means for acquiring position information of a user, means for transmitting user data including the acquired position information to a server, means for analyzing regional trends based on the user data by the server, means for inputting an analysis result into a generation model to generate a personalized advertisement or regional information, and means for transmitting the generated advertisement or regional information to a user terminal or an electric bulletin board.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern cities and tourist destinations, users are faced with the challenge of finding the information they need due to the sheer volume of information and advertisements provided to them. Furthermore, personalized information provision based on users' interests and preferences is insufficient, and inconsistent information is often presented from multiple sources. This makes it difficult for users to obtain useful information. Advertisers also face the challenge of being unable to effectively deliver advertisements to their targets. [Means for solving the problem]

[0005] This invention provides personalized advertisements and local information tailored to a user's current location using a system that includes a means for acquiring user location information and transmitting it to a server. The system also includes a means for the server to analyze local trends based on user data and input the analysis results into a generative model to generate advertisements and local information. The generated advertisements and information are sent to user terminals and electronic billboards. The system also includes a means for recording the user's behavior when interacting with advertisements and information and using that data to generate the next personalized advertisement. In this way, useful information is provided to users in real time, which can contribute to revitalizing local economies and promoting tourism.

[0006] "User" refers to the end-user who provides location, interest, preference, and behavioral data.

[0007] "Location Information" refers to GPS coordinate data that indicates a user's current location.

[0008] "User Data" refers to data including, but not limited to, your location, interests and preferences, behavioral data, and other related information.

[0009] "Server" refers to the computer system that collects and analyzes user data, and generates and delivers advertisements.

[0010] "Local Trends" refers to popular activities and events based on user interests and behavior in a given area.

[0011] A "generative model" refers to an algorithm that uses AI technology to generate personalized advertisements and local information from the results of data analysis.

[0012] "Advertising" means any message or information promoting a product or service to a user.

[0013] "Local information" refers to information about events, tourist attractions, stores, etc. related to the area where the user is currently located.

[0014] "Terminal" refers to smartphones and other digital devices used by users, as well as display devices such as electronic bulletin boards.

[0015] "Electronic bulletin board" refers to an electronic display device installed in a public place for displaying information.

[0016] "Interaction" refers to actions such as tapping or clicking that a user takes on an advertisement or information displayed on a device or electronic billboard.

[0017] "Behavioral data" refers to records of a user's operation history and information of interest while using the service.

[0018] "Personalization" refers to customizing certain information based on a user's individual interests and preferences. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] MODE FOR CARRYING OUT THE INVENTION

[0041] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. This system works through a server, terminals, and records of user interactions.

[0042] Retrieving User Information

[0043] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[0044] Big Data Analysis

[0045] The server analyzes the collected user data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[0046] Ad generation by generative AI

[0047] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history.

[0048] Real-time information distribution

[0049] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by.

[0050] Recording user interactions

[0051] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided.

[0052] Specific examples

[0053] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The generation AI uses this information to create an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, coupons for nearby restaurants are also generated and provided at the same time based on the user's past usage history. The user receives this information, and the data on their interactions is sent back to the server to be used to provide information in the future.

[0054] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[0055] The processing flow will be explained below.

[0056] Specific processing steps of the program

[0057] Information Acquisition Phase

[0058] Step 1:

[0059] The user launches the smartphone app and logs in.

[0060] The device uses GPS to obtain the user's current location.

[0061] Step 2:

[0062] The location information acquired by the device is sent to the server.

[0063] The server receives the location information and determines the user's location.

[0064] Step 3:

[0065] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[0066] The device sends this selection data to the server.

[0067] Step 4:

[0068] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[0069] The server receives the behavioral data and stores it in a database.

[0070] Data analysis phase

[0071] Step 5:

[0072] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[0073] The server also collects social media data and local pedestrian flow data.

[0074] Step 6:

[0075] The server cleanses the data and formats it into an easy-to-parse format.

[0076] Incomplete or incorrect data is filtered out.

[0077] Step 7:

[0078] The server analyzes regional trends.

[0079] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[0080] Ad generation phase

[0081] Step 8:

[0082] The server inputs the analysis results into the generative model.

[0083] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, and trend data.

[0084] Step 9:

[0085] The server utilizes the generative model to generate personalized advertisements or local information.

[0086] For example, if a user is near a museum, ads for art exhibitions will be generated.

[0087] Step 10:

[0088] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[0089] The server finalizes the advertisement and information.

[0090] Information distribution phase

[0091] Step 11:

[0092] The server sends the generated advertisements and local information to the user's terminal.

[0093] The device will display information to the user via push notifications or at specific times.

[0094] Step 12:

[0095] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[0096] Devices and electronic billboards display information relevant to the user's current location.

[0097] Interaction Recording Phase

[0098] Step 13:

[0099] The device records how the user interacts with advertisements and information.

[0100] For example, if a user taps on a push notification to view more information, that action is recorded.

[0101] Step 14:

[0102] The terminal transmits the recorded interaction data to the server.

[0103] The server receives the interaction data and stores it in a database.

[0104] Step 15:

[0105] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[0106] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[0107] Through these steps, users will be provided with personalized advertisements and local information in real time, creating a system that will contribute to revitalizing the local economy and promoting tourism.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] Conventional ad delivery systems are unable to fully utilize users' real-time interests, preferences, location information, and behavioral data, making it difficult to provide truly useful personalized ads and local information to users. Furthermore, there is a lack of technology to effectively analyze this data and provide information at the optimal time. Furthermore, there are few ways to improve the accuracy of subsequent ads and information based on user responses.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes: means for acquiring user location information; means for transmitting user data, including the acquired location information and interest and preference data, to the server; means for transmitting user behavior data from the terminal to the server; means for the server to analyze regional trends based on the user data and external data using a big data analysis tool; means for inputting the analysis results into a generative AI model to generate personalized advertisements or regional information; means for transmitting the generated advertisements or regional information to the user terminal or display device; means for recording the user's behavior when interacting with the generated advertisements or regional information and transmitting the behavior data to the server; and means for the server to analyze the interaction data and use it to generate the next personalized advertisement or regional information. This makes it possible to effectively utilize the user's real-time location information, interest and preference data, and behavior data to provide personalized information. Furthermore, the accuracy of the service can be further improved based on user feedback.

[0113] "User" refers to an individual or entity that uses the System, specifically the entity that provides location information, interest and preference data, and behavioral data.

[0114] "Location Information" means data that indicates a user's current geographic location, obtained using technologies such as GPS.

[0115] "Interest and Preference Data" refers to information about your interests and preferences that you enter within the App.

[0116] "Behavioral data" refers to data including the history of actions taken by a user within an app, such as search history and browsing history.

[0117] "Server" refers to a computer system that receives, stores, and analyzes data sent by users, and generates and distributes advertisements and local information.

[0118] "Big data analytics tools" are software and technologies that allow for the efficient analysis of vast amounts of data, specifically those used to identify regional trends and demand.

[0119] "Generative AI model" refers to artificial intelligence technology that uses pre-trained algorithms to generate personalized ads and local information based on prompts.

[0120] A "prompt" is an instruction entered into a generative AI model, containing a specific request to generate a particular advertisement or local information.

[0121] "User terminal" refers to a device such as a smartphone or tablet used by a user, which obtains location information and receives advertisements.

[0122] "Display device" refers to a device such as an electronic signboard or display for displaying information from the system.

[0123] "Interaction Data" is data that indicates how a user interacts with an advertisement or local information and is sent to a server.

[0124] MODE FOR CARRYING OUT THE INVENTION

[0125] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. The system functions through a server, terminals, and records of user interactions.

[0126] Retrieving User Information

[0127] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The user can also input their interests and preferences within the app, and this interest and preference data is also sent to the server. Furthermore, user behavior data, such as search history and browsing history within the app, is also collected and sent to the server.

[0128] Big Data Analysis

[0129] The server uses big data analysis tools (e.g., Apache® Hadoop) to identify local trends and demand based on the received user data. Analysis includes social media mention counts and foot traffic data. The results of this analysis are fed into a generative AI model (e.g., OpenAI® GPT-4®).

[0130] Ad generation by generative AI

[0131] The server creates a prompt based on the analysis results and inputs it into the generative AI model. For example, a prompt such as "Please generate an advertisement for a popular art event in Kyoto" is input into the generative AI model. The generative AI model generates personalized advertisements and local information based on this prompt.

[0132] Real-time information distribution

[0133] The generated advertisements and local information are sent from the server to the user's smartphone via push notifications. Using communication technologies such as Firebase Cloud Messaging, the information is delivered to the user instantly. The information is also displayed on display devices (e.g., electronic billboards) when the user passes nearby.

[0134] Recording user interactions

[0135] When a user interacts with an advertisement or information, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is generated.

[0136] Specific examples

[0137] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, the app uses GPS to obtain their current location (Kyoto) and sends it to the server. The server then analyzes social media data and people flow data to determine that the "Kyoto Art Festival" is popular. The server then inputs a prompt message, "Please generate an advertisement for the Kyoto Art Festival," into the generative AI model, which then generates an advertisement for the festival. The generated advertisement is sent from the server to the user's smartphone via push notification, allowing the user to receive the information immediately. In addition, similar information is displayed in real time on nearby electronic billboards. When the user taps the push notification to view more information, their behavioral data is sent to the server and used to generate future advertisements.

[0138] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Step 1:

[0141] Retrieving User Information

[0142] When a user launches a smartphone app, the device uses GPS to obtain current location information. This location information is then sent from the device to a server. Specifically, the device receives input data from the GPS and sends it to the server in real time. The output is location data.

[0143] Step 2:

[0144] Enter your interests and preferences

[0145] The user enters their interests and preferences within the app. This data is also sent to the server. Specifically, the device receives the user's input as text data and sends it to the server. The output is interest and preference data.

[0146] Step 3:

[0147] Behavioral data collection

[0148] Every time a user searches or browses within the app, the device records the behavioral data and sends it to the server. Specifically, the device captures the user's action events (search history, browsing history, etc.) and sends them to the server. The output is behavioral data.

[0149] Step 4:

[0150] Data collection

[0151] The server receives location information, interest and preference data, and behavioral data. Specifically, the server acquires this data from each device and stores it in a database. The input is the data sent from the device, and the output is organized user data.

[0152] Step 5:

[0153] Big Data Analysis

[0154] The server uses big data analysis tools (e.g., Apache Hadoop) to identify regional trends and demand. The server analyzes data such as the number of mentions on social media and foot traffic data to extract trend information for each region. The input is user data and external data, and the output is the analysis results.

[0155] Step 6:

[0156] Generate prompt statement

[0157] The server creates a prompt based on the analysis results. Specifically, it generates a text-based instruction to be input into the generative AI model. For example, a prompt might be generated saying, "Please generate an advertisement for a popular art event in Kyoto." The input is the analysis results, and the output is the prompt.

[0158] Step 7:

[0159] AI-powered ad generation

[0160] The prompt text generated by the server is input into a generative AI model (e.g., OpenAI GPT-4) to generate personalized advertisements and local information. Specifically, the AI ​​analyzes the prompt text and generates advertisements and information based on it. The input is the prompt text, and the output is the generated advertisements and local information.

[0161] Step 8:

[0162] Real-time information distribution

[0163] The server sends the generated ads and location information to the user's smartphone via push notification. The information is delivered in real time using Firebase Cloud Messaging. The input is the generated ads and location information, and the output is the push notification.

[0164] Step 9:

[0165] Electronic bulletin board update

[0166] The server sends information to a nearby display device, which displays relevant advertisements and information as the user passes by. The input is the generated advertisement or information, and the output is the display on the display device.

[0167] Step 10:

[0168] Recording user interactions

[0169] Every time a user taps on a push notification to view more information, the device records that action and sends it to the server. The input is the user interaction, and the output is the recorded behavior data.

[0170] Step 11:

[0171] Utilizing data to generate the next ad

[0172] The server analyzes the interaction data and uses it to generate the next personalized advertisement and local information. Specifically, it generates more accurate prompts and advertisements based on past user behavior data. The input is the interaction data, and the output is the analysis results that will be used to generate the next advertisement.

[0173] (Application example 1)

[0174] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0175] While modern autonomous vehicles have advanced in technology to improve safety and efficiency, they still face limitations in providing personalized information and entertainment to passengers. In particular, in tourism and commercial use, there is a lack of ways for passengers to receive information tailored to their interests and preferences in real time to enhance their experience. There is also a need for a mechanism to utilize user feedback on the information provided to provide more highly personalized services.

[0176] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0177] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information to the server, means for the server to analyze regional trends based on the user data, means for inputting the analysis results into a generative AI model to generate personalized tourist spot information or advertisements, means for transmitting the generated tourist spot information or advertisements to a user terminal or display device in the vehicle, means for recording user behavior when interacting with the generated tourist spot information or advertisements and transmitting the behavioral data to the server, and means for the server to analyze the interaction data and use it to generate the next personalized tourist spot information or advertisements. This allows passengers to receive personalized information in real time while in an autonomous vehicle, improving their experience and continuously improving the accuracy of the information provided.

[0178] An "autonomous vehicle" is a vehicle that does not require a driver and operates using an automatic control system.

[0179] "Location Information" means data that indicates a specific location using GPS or other location measurement technology.

[0180] "User Data" refers to data that includes general information about a user, such as the user's location, interests, preferences, and behavioral data.

[0181] "Server" means a computer system that collects, analyzes, processes data, and distributes the generated information.

[0182] "Regional trends" refers to information about events, places, and other things that are currently popular or attracting attention in a particular region.

[0183] "Analysis" is the process of extracting useful information and patterns from collected data.

[0184] A "generative AI model" is an algorithm that uses artificial intelligence to generate personalized information from input data.

[0185] "Tourist spot information" refers to detailed information about tourist spots.

[0186] An "advertisement" is an informational message created to promote a particular product, service, etc.

[0187] A "user device" is a device that can be directly operated by a user, such as a smartphone or tablet.

[0188] A "display device" is a display or screen used to visually present information to a user.

[0189] "Interact" means that a user interacts in some way with the information or application provided.

[0190] "Behavioral Data" means data that describes how a user interacts with an application or the information provided to them.

[0191] This invention aims to realize a system for providing personalized information to passengers in an autonomous vehicle. Specifically, the system is constructed by combining the following steps.

[0192] First, the server uses GPS to obtain the location information of the autonomous vehicle. The GPS module is installed in the vehicle and periodically determines its current location. This location information is sent to the server.

[0193] The server then obtains passengers' interests and preferences through user devices such as tablets and smartphones. Passengers input their interests and preferences through tablets installed in the vehicle or through an app on their own smartphone. This data is also sent to the server and stored as user data.

[0194] The server also collects behavioral data, such as user search and browsing history within the app. Based on this data, the server analyzes trends and demand by region. A big data analysis system (e.g., Apache Spark) is used for the analysis, and the popularity and key points of events in a specific region are identified based on the number of mentions on social media.

[0195] The server then inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate personalized tourist spot information and advertisements. An example of a specific prompt might be, "The user is interested in historical buildings. Please tell me about popular historical buildings as the latest tourist attractions near their current location."

[0196] The generated tourist information and advertisements are sent in real time to tablets and smartphones on board, giving passengers instant access to personalized information.

[0197] Finally, when a passenger interacts with the displayed information (e.g., tapping on a coupon), their actions are again sent to the server, which analyzes this interaction data and uses it the next time the information is provided, further improving the accuracy and personalization of the information provided.

[0198] For example, if a passenger expresses an interest in "historical buildings," the server analyzes local trends based on the acquired location information and generates information and coupons related to "historical castles," a popular tourist spot nearby. This information is displayed in real time on the tablet, and when the passenger uses the coupon, the data is sent back to the server to help personalize the service for future visits.

[0199] In this way, the invention can improve the passenger experience in autonomous vehicles and continuously improve the accuracy of the information provided.

[0200] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0201] Step 1:

[0202] (Processing content) Acquisition and transmission of location information

[0203] (Specific operation) The GPS module installed in the autonomous vehicle periodically acquires its current location. This location information is sent to a server via a communication module in the vehicle.

[0204] (Input) Current position from the GPS module

[0205] (Data processing) Format conversion of location information

[0206] (Output) Location data sent to the server

[0207] Step 2:

[0208] (Processing content) Acquiring user interests and preferences

[0209] (Specific operation) The user inputs their interests and preferences using a tablet installed in the vehicle or an app on the passenger's smartphone. The input data is sent from the app to the server.

[0210] (Input) Interest and preference data from tablet and smartphone apps

[0211] (Data processing) Linking user ID with interest and preference data

[0212] (Output) User interest and preference data sent to the server

[0213] Step 3:

[0214] (Processing content) Collection of user behavior data

[0215] (Specific operation) Collects behavioral data such as search history and browsing history within the app and sends it to the server. To do this, the app generates log data and periodically sends it to the server.

[0216] (Input) Search history and browsing history within the app

[0217] (Data processing) Standardizing the format of behavioral data

[0218] (Output) Action log data sent to the server

[0219] Step 4:

[0220] (Processing content)Analysis of regional trends

[0221] (Specific operation) The server uses the collected user data and location information to analyze regional trends using a big data analysis system (e.g., Apache Spark). Trends are identified based on data such as the number of mentions on social media and the flow of people.

[0222] (Input) User data, location information

[0223] (Data processing) Big data aggregation and statistical analysis

[0224] (Output) Regional trend data as analysis results

[0225] Step 5:

[0226] (Processing content) Creation of personalized information

[0227] (Specific operation) The server inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate tourist spot information and advertisements appropriate for the user. A prompt sentence is input into the generative AI model, and it is generated based on this.

[0228] (Input) Regional trend data, prompt statement

[0229] (Data processing) Information generation using generative AI models

[0230] (Output) Personalized information (tourist spot information, advertisements)

[0231] Step 6:

[0232] (Processing content) Real-time information distribution

[0233] (Specific operation) The generated personalized information is sent to tablets and smartphones in the vehicle and notifications are sent, allowing passengers to receive information in real time.

[0234] (Input) Personalization Information

[0235] (Data processing) Creation and sending of notification messages

[0236] (Output) Information displayed on the user's terminal

[0237] Step 7:

[0238] (Processing content) Recording of user interactions

[0239] (Specific actions) When the user interacts with the displayed information (tap, click, use, etc.), that action is sent back to the server, which stores this data and uses it for the next analysis.

[0240] (Input) User interaction data

[0241] (Data processing) Recording interaction data and integrating it into analytical data

[0242] (Output) Interaction data stored on the server

[0243] Step 8:

[0244] (Processing content) Analysis for the next information provision

[0245] (Specific operation) The server analyzes the interaction data and uses it to generate personalized information for the next visit. This analysis makes it possible to provide more accurate information.

[0246] (Input) Interaction data

[0247] (Data processing) Modify the model for the next generation of personalized information

[0248] (Output) Analysis data for the next information provision

[0249] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0250] MODE FOR CARRYING OUT THE INVENTION

[0251] This invention is a system that provides personalized advertisements and local information in real time using an emotion engine that recognizes user emotions in addition to user location information, interests, preferences, and behavioral data. This system functions through a server, terminals, the emotion engine, and records of user interactions.

[0252] Retrieving User Information

[0253] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[0254] Acquiring emotion data

[0255] The device is equipped with an emotion engine that analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data, which is then sent to a server in real time and stored as the user's emotional profile.

[0256] Big Data Analysis

[0257] The server analyzes the collected user data and sentiment data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[0258] Ad generation by generative AI

[0259] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history and current emotional state.

[0260] Real-time information distribution

[0261] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by. In addition, the timing of information delivery can be adjusted based on the user's emotions recognized by the emotion engine. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[0262] Recording user interactions

[0263] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided. Emotional data evoked by the user's interaction is also recorded and sent to the server as feedback.

[0264] Specific examples

[0265] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The emotion engine then recognizes the user's relaxed emotions. Based on this information, the generative AI creates an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, based on the user's past usage history and current emotional state, coupons for nearby restaurants are also generated and provided at the same time. The user receives this information, and the data on their interactions is sent back to the server to be used for providing information in the future.

[0266] In this way, the system can provide users with useful information in real time, contributing to regional economic revitalization and tourism promotion. It can also continuously collect user feedback to improve the accuracy of the information provided. The introduction of an emotion engine enables more advanced personalization based on the user's emotional state, further improving the effectiveness of information provision.

[0267] The processing flow will be explained below.

[0268] Specific processing steps of the program

[0269] Information Acquisition Phase

[0270] Step 1:

[0271] The user launches the smartphone app and logs in.

[0272] The device uses GPS to obtain the user's current location.

[0273] Step 2:

[0274] The location information acquired by the device is sent to the server.

[0275] The server receives the location information and determines the user's location.

[0276] Step 3:

[0277] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[0278] The device sends this selection data to the server.

[0279] Step 4:

[0280] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[0281] The server receives the behavioral data and stores it in a database.

[0282] Emotion data acquisition phase

[0283] Step 5:

[0284] The device uses a camera and microphone to detect the user's facial expressions and tone of voice.

[0285] The emotion engine analyzes this data to recognize the user's emotions.

[0286] Step 6:

[0287] The recognized emotion data is sent to the server in real time.

[0288] The server receives the emotion data and adds it to the user profile.

[0289] Data analysis phase

[0290] Step 7:

[0291] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[0292] The server also collects social media data and local pedestrian flow data.

[0293] Step 8:

[0294] The server cleanses the data and formats it into an easy-to-parse format.

[0295] Incomplete or incorrect data is filtered out.

[0296] Step 9:

[0297] The server analyzes regional trends.

[0298] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[0299] Ad generation phase

[0300] Step 10:

[0301] The server inputs the analysis results into the generative model.

[0302] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, emotional data, and trend data.

[0303] Step 11:

[0304] The server utilizes the generative model to generate personalized advertisements or local information.

[0305] For example, if a user is near a museum, ads for art exhibitions will be generated.

[0306] Step 12:

[0307] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[0308] The server finalizes the advertisement and information.

[0309] Information distribution phase

[0310] Step 13:

[0311] The server sends the generated advertisements and local information to the user's terminal.

[0312] The device will display information to the user via push notifications or at specific times.

[0313] Step 14:

[0314] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[0315] Devices and electronic billboards display information relevant to the user's current location.

[0316] Step 15:

[0317] Based on the emotional data, the server adjusts the timing of information distribution.

[0318] For example, it could deliver restaurant promotion information when the user is relaxing.

[0319] Interaction Recording Phase

[0320] Step 16:

[0321] The device records how the user interacts with advertisements and information.

[0322] For example, if a user taps on a push notification to view more information, that action is recorded.

[0323] Step 17:

[0324] The terminal transmits the recorded interaction data to the server.

[0325] The server receives the interaction data and stores it in a database.

[0326] Step 18:

[0327] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[0328] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[0329] Through these steps, a system will be built that provides users with personalized advertisements and local information in real time, contributing to the revitalization of local economies and the promotion of tourism. Furthermore, the introduction of an emotion engine will provide more appropriate information based on the user's emotional state, further improving the effectiveness of information provision.

[0330] Example 2

[0331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0332] Conventional personalized advertising and local information provision systems typically generate advertisements and information based on users' location information, interests, preferences, and behavioral data. However, personalization that takes into account the user's emotional state is not adequate, and the timing and content are often inappropriate. As a result, there are issues with not being able to attract users' attention and reducing the effectiveness of advertisements and information.

[0333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0334] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information, interests, preferences, and behavioral data to the server, means for acquiring user emotion data using an emotion engine installed in the terminal, means for transmitting the acquired emotion data to the server, means for the server to analyze regional trends based on the user data and emotion data and generate a prompt message using a generative AI model, means for generating personalized advertisements or regional information based on the generated prompt message, means for transmitting the generated advertisements or regional information to the user terminal or electronic billboard, and means for adjusting the timing of information delivery based on the user's emotional state, thereby enabling the provision of more effective personalized advertisements and regional information according to the user's emotional state.

[0335] "Location information" is geographic coordinate data that indicates where a user is currently located.

[0336] "Interests and Preferences" refers to information about a user's preferred activities, places, items, etc.

[0337] "Behavioral data" refers to information such as search history, browsing history, and click history when a user uses the system.

[0338] "Emotional data" refers to information about a user's emotional state, analyzed from facial expressions, tone of voice, etc.

[0339] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[0340] A "server" is a remote computer system that receives and analyzes user data and emotional data.

[0341] A "generative AI model" is a machine learning model that generates prompts based on user data and emotional data to create personalized advertisements and information.

[0342] A "prompt sentence" is an input sentence that a generative AI model uses to generate personalized ads or information.

[0343] An "advertisement" is text, image, or video that provides commercial information to users.

[0344] "Local information" is information about events and places associated with a particular geographic area.

[0345] A "push notification" is a notification message that provides information to a user's device in real time.

[0346] An "electronic billboard" is an electronic display device for displaying advertisements and information in a specific area.

[0347] "Interact" refers to the user's reaction to and manipulation of the advertisements and information provided.

[0348] "Interaction data" is a record of the actions and behaviors that users take in response to advertisements or information.

[0349] "Feedback" is information used to improve the system's personalization accuracy based on user emotional and interaction data.

[0350] This invention is a system that uses user location information, interests, preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time. This system is mainly implemented using a server, a terminal, an emotion engine, and a generative AI model. Detailed embodiments of the system are described below.

[0351] Retrieving User Information

[0352] When a user launches the smartphone app, the device uses GPS to obtain its current location information, which is then captured as latitude and longitude data and sent to the server in JSON format.

[0353] Next, users enter their interests and preferences within the app. For example, they enter information such as their interest in museums and restaurants in text format, and this data is also sent to the server. In addition, the device collects behavioral data such as search history, browsing history, and click history within the app, and this information is also sent to the server in JSON format.

[0354] Acquiring emotion data

[0355] The device is equipped with an emotion engine that captures the user's emotional data. The emotion engine uses a camera to capture the user's facial expressions and analyzes their emotional state using a facial recognition algorithm. It also uses a microphone to capture the tone of voice and analyzes it using a voice recognition algorithm. The captured emotion data is sent to a server in real time and stored as the user's emotional profile.

[0356] Big Data Analysis

[0357] The server integrates the acquired user data and sentiment data to analyze trends and demand in each region. Specifically, it analyzes big data including the number of mentions on social media and population flow data. This analysis can, for example, identify the popularity of events in a specific region and identify new points of interest.

[0358] Ad generation by generative AI

[0359] The server inputs the analysis results into a generative AI model to generate personalized advertisements and local information. For example, if a user is interested in museums, the generative AI model will generate advertisements based on the following prompt:

[0360] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[0361] The generative AI model uses this prompt to create the most relevant ads and local information for the user.

[0362] Real-time information distribution

[0363] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access the information instantly. On electronic billboards, relevant information is displayed in real time as the user passes by. The emotion engine also adjusts the timing of information delivery based on the user's emotional state. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[0364] Recording user interactions

[0365] When a user interacts with an advertisement or information provided through an app or electronic billboard, that behavior is recorded and sent to a server. For example, if a user taps on a push notification to view more information, that behavioral data is sent to the server. This interaction data is used to provide advertisements and information in the future, enabling more accurate personalization.

[0366] In this way, the system integrates users' location information, interests and preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time, enabling it to accurately deliver useful information to users.

[0367] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0368] Step 1:

[0369] Retrieving User Information

[0370] The device uses GPS to obtain the user's current location information. The input data is the latitude and longitude received from the GPS sensor. The output data is the obtained location information (latitude and longitude) converted to JSON format.

[0371] The device sends this location information to the server. Specifically, the device sends the location information to the server using an HTTP POST request.

[0372] Step 2:

[0373] Enter your interests and preferences

[0374] Users input their interests and preferences into a smartphone app. The input data is information about interests and preferences that users enter by tapping or entering text.

[0375] The device converts this information into JSON format and sends it to the server. The output data is JSON format data of the user's interests and preferences. Specifically, the device collects data entered through the user interface and sends it to the server.

[0376] Step 3:

[0377] Behavioral data collection

[0378] The device collects the user's search history, browsing history, and click history within the app. The input data is a log of the user's actions.

[0379] The device converts this data into JSON format and sends it to the server. The output data is JSON formatted behavioral data. Specifically, it monitors the action log and sends the data to the server in real time. As with other data, it uses an HTTP POST request.

[0380] Step 4:

[0381] Acquiring emotion data

[0382] The emotion engine installed on the device analyzes the user's facial expressions. The input data is image data of the user's face captured by the camera. The output data is data indicating the analyzed emotional state (e.g., smiling, angry, surprised, etc.).

[0383] The device sends this data to a server, which uses a facial recognition algorithm to analyze the user's emotional state and sends the results to the server via an HTTP POST request.

[0384] Step 5:

[0385] Voice Tone Analysis

[0386] The device analyzes the tone of the user's voice. The input data is the voice data captured by the microphone. The output data is data indicating the analyzed emotional state (e.g., stressed state, relaxed state, etc.).

[0387] The device sends this data to the server, which uses a speech recognition algorithm to analyze the tone of the voice and sends the results to the server via an HTTP POST request.

[0388] Step 6:

[0389] Big Data Analysis

[0390] The server integrates the received user data and sentiment data to analyze local trends and demand. The input data is the collected user data and sentiment data. Additional big data (e.g., social media mention counts and population mobility data) is also used for analysis.

[0391] The server analyzes this data to identify trends and demand in specific regions. The output data is data showing trends and demand by region. Specifically, it runs big data analysis algorithms to identify trends by region.

[0392] Step 7:

[0393] Ad generation by generative AI

[0394] The server generates prompt sentences for the generative AI model based on the results of big data analysis. The input data is the analysis results, the user's interests and preferences, behavioral data, and emotional data. Examples of prompt sentences are:

[0395] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[0396] The generative AI model generates personalized advertisements and local information based on this prompt. The output data is personalized advertisements and local information. Specifically, the generative AI model analyzes the prompt and generates appropriate advertisements and information.

[0397] Step 8:

[0398] Real-time information distribution

[0399] The server transmits the generated advertisement and area information to the user terminal or the electronic billboard, and the input data is the generated advertisement and area information.

[0400] The server delivers this data in real time, providing information to users via push notifications on smartphones and real-time displays on electronic billboards. Specifically, the server sends the generated advertisements and local information to the delivery system, triggering push notifications.

[0401] Step 9:

[0402] Recording user interactions

[0403] When a user interacts with the advertisements or information provided, their actions are recorded. The input data is a log of the user's interaction (for example, tapping on a push notification to view more information).

[0404] The device sends this interaction data to the server. The output data is an interaction log in JSON format. Specifically, the device detects interaction events and sends the data to the server in real time.

[0405] Step 10:

[0406] Data Feedback

[0407] The server analyzes the user's interaction data and uses it to generate personalized ads and local information for the next visit. The input data is the collected interaction data.

[0408] The server analyzes this data to identify emerging trends and patterns. The output is an optimization profile for the next ad or information offering. Specifically, machine learning algorithms are used to analyze the interaction data and use the results to improve the accuracy of future ads and information offerings. Emotional feedback is also included in the analysis.

[0409] (Application example 2)

[0410] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0411] Conventional advertising systems have a low level of personalization accuracy, making it difficult to provide users with timely and useful information. Furthermore, delivering ads uniformly without considering the user's current emotional state can be counterproductive. For example, if an ad is sent to a user when they are tired, it may not only not pique their interest, but may also create a negative impression of the ad itself. Furthermore, there is no established method for effectively utilizing real-time location information and emotional data, making it difficult to provide information tailored to local trends and users' interests and preferences.

[0412] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for transmitting user data and emotion data to the server, means for analyzing regional trends based on the user data and emotion data, and means for inputting the analysis results into a generative AI model and generating personalized advertisements or regional information based on the user's current location and emotional state. This makes it possible to provide timely and highly accurate personalized information that takes into account the user's current location and emotional state.

[0413] "User Data" is a general term for all data related to a user, such as the user's location information, interests and preferences, and behavioral history.

[0414] "Emotional data" is information about a user's emotional state at any given time, obtained by analyzing their facial expressions, tone of voice, and other biometric signals.

[0415] A "server" is a computer system that processes various data, such as analyzing user data and emotional data, generating advertisements, and providing information.

[0416] "Local trends" refers to information that shows trends and patterns regarding user interests, popular events, facilities, etc. within a particular geographic area.

[0417] "Generative AI models" refer to artificial intelligence algorithms and learning models used to generate personalized ads and location information using user and emotional data as input.

[0418] "Personalized advertising or location information" refers to advertising or location information that is individually optimized based on a user's individual interests, preferences, behavioral history, and emotional state.

[0419] "User terminal" refers to a communication device used by a user, such as a smartphone or tablet.

[0420] "Display device" refers to a device, such as an electronic bulletin board or digital signage, that visually presents advertisements or information to users.

[0421] "Push notification" is a messaging service that sends information from a server to a user's device in real time.

[0422] "Interact" means that a user interacts in some way with the advertisement or location information provided.

[0423] This invention is a system that collects and analyzes user location information, interests, preferences, behavioral data, and emotional data, and provides personalized advertisements and local information in real time. This system operates mainly using the following hardware and software.

[0424] Hardware

[0425] 1. User device: a smartphone or tablet, including a GPS and a camera.

[0426] 2. Display devices: digital signage and electronic bulletin boards.

[0427] software

[0428] 1. Emotion Engine: Facial expression recognition and voice analysis software to capture the user's emotional state.

[0429] 2. Generative AI model: Runs on the server and generates personalized ads and local information based on user data and emotional data.

[0430] 3. Data transmission / reception module: A communication module for exchanging data between the user terminal and the server.

[0431] Process Overview

[0432] 1. Acquisition and transmission of user data

[0433] When a user launches a smartphone app, the device's GPS function is used to obtain current location information. The app also collects the user's interests, preferences, and behavioral history. This data is sent to a server in real time.

[0434] 2. Acquiring Emotion Data

[0435] The smartphone is equipped with an emotion engine that analyzes the user's facial expressions and tone of voice, and this acquires emotional data, which is also sent to the server.

[0436] 3. Data analysis and ad generation

[0437] On the server side, user and sentiment data is analyzed to understand specific local trends and user interests, which is then used by generative AI models to generate personalized ads and local information.

[0438] 4. Provision of Information

[0439] The generated advertisements and local information are sent in real time to the user's device or display device, and on smartphones, the information is provided via push notifications.

[0440] Specific examples

[0441] For example, if a user is in a shopping mall, the generative AI model can generate special sale information and coupons for the mall based on the smartphone's location information, collected interest and preference data, and the relaxed state detected by the emotion engine, and provide them to the user via push notification.

[0442] Prompt Sentence Examples

[0443] "Use up-to-date sentiment data to generate ads that are best suited to relaxed users in a specific location."

[0444] This will enable us to provide users with the information they need in a timely manner, improving the effectiveness of our advertising. In addition, by collecting user feedback data, we will be able to further improve the accuracy of the information we provide next time.

[0445] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0446] Step 1:

[0447] When a user launches a smartphone app, the device uses its GPS function to obtain the user's current location. This location information is input data that the application sends to the server as user data. This location information is used to inform the server of the user's current location. Specifically, the smartphone's GPS sensor obtains the location information, and the application sends it to the server via an HTTP request.

[0448] Step 2:

[0449] The user's interests, preferences, and past behavioral data are also collected at the same time. This information, such as the user's past search keywords and visited places, is sent to the server along with the interest and preference data entered within the app. This input data forms an individual user profile and is used by the server to analyze the user's interests. Specifically, past history data is collected from the application's database and sent to the server along with location information.

[0450] Step 3:

[0451] The device is equipped with an emotion engine that analyzes the user's facial expressions (using the camera) and voice tone (using the microphone) to obtain emotion data. This emotion data is also sent to the server. The emotion data as input data indicates the user's current emotional state and is used to personalize advertisements. Specifically, the device uses the smartphone's camera and microphone to collect the user's biometric data, and the results analyzed by the emotion engine are sent to the server.

[0452] Step 4:

[0453] The server analyzes the received location information, interest and preference data, behavioral data, and emotional data. For this analysis, it uses big data analytics technology to gain insights based on local trends and user interests. This process includes data processing and calculation, such as data retrieval from the database, cleansing, point analysis, and trend analysis. Specifically, it retrieves trend data from the database and extracts information on popular events and stores by region.

[0454] Step 5:

[0455] The server inputs the analysis results into a generative AI model to generate personalized ads or local information based on the user's current location and emotional state. In this generation process, the generative AI model generates the optimal ads or information based on the input data and generates the results. Specifically, the generative AI model matches the user's profile with trend data to generate the optimal ads or information. An example of a prompt is, "Use the latest emotional data to generate the optimal ads for relaxed users in a specific area."

[0456] Step 6:

[0457] The generated advertisements and local information are sent from the server to the user's device or display device. In the case of smartphones, the information is provided to the user via push notification. The generated results as input data are sent to the device as an HTTP response, and the device displays the push notification. Specifically, the server returns the generated results as an HTTP response, and the smartphone receives the data and triggers the push notification.

[0458] Step 7:

[0459] When a user interacts with the generated advertisement or local information, the behavioral data is recorded and sent to the server. This behavioral data includes the click history of push notifications and the time spent viewing advertisements. Specifically, the system captures user operation events and sends them to the server as behavioral data.

[0460] Step 8:

[0461] The server analyzes the interaction data and uses it to generate the next personalized advertisements and local information. This analysis process includes analyzing past behavioral patterns and evaluating emotional data. Specifically, the behavioral history stored in the database is analyzed and used as input data for the next generation AI model.

[0462] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0463] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0464] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0465] [Second embodiment]

[0466] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0467] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0468] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0469] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0470] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0471] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0472] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0473] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0474] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0475] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0476] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0477] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0478] MODE FOR CARRYING OUT THE INVENTION

[0479] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. This system works through a server, terminals, and records of user interactions.

[0480] Retrieving User Information

[0481] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[0482] Big Data Analysis

[0483] The server analyzes the collected user data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[0484] Ad generation by generative AI

[0485] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history.

[0486] Real-time information distribution

[0487] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by.

[0488] Recording user interactions

[0489] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided.

[0490] Specific examples

[0491] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The generation AI uses this information to create an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, coupons for nearby restaurants are also generated and provided at the same time based on the user's past usage history. The user receives this information, and the data on their interactions is sent back to the server to be used to provide information in the future.

[0492] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[0493] The processing flow will be explained below.

[0494] Specific processing steps of the program

[0495] Information Acquisition Phase

[0496] Step 1:

[0497] The user launches the smartphone app and logs in.

[0498] The device uses GPS to obtain the user's current location.

[0499] Step 2:

[0500] The location information acquired by the device is sent to the server.

[0501] The server receives the location information and determines the user's location.

[0502] Step 3:

[0503] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[0504] The device sends this selection data to the server.

[0505] Step 4:

[0506] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[0507] The server receives the behavioral data and stores it in a database.

[0508] Data analysis phase

[0509] Step 5:

[0510] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[0511] The server also collects social media data and local pedestrian flow data.

[0512] Step 6:

[0513] The server cleanses the data and formats it into an easy-to-parse format.

[0514] Incomplete or incorrect data is filtered out.

[0515] Step 7:

[0516] The server analyzes regional trends.

[0517] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[0518] Ad generation phase

[0519] Step 8:

[0520] The server inputs the analysis results into the generative model.

[0521] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, and trend data.

[0522] Step 9:

[0523] The server utilizes the generative model to generate personalized advertisements or local information.

[0524] For example, if a user is near a museum, ads for art exhibitions will be generated.

[0525] Step 10:

[0526] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[0527] The server finalizes the advertisement and information.

[0528] Information distribution phase

[0529] Step 11:

[0530] The server sends the generated advertisements and local information to the user's terminal.

[0531] The device will display information to the user via push notifications or at specific times.

[0532] Step 12:

[0533] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[0534] Devices and electronic billboards display information relevant to the user's current location.

[0535] Interaction Recording Phase

[0536] Step 13:

[0537] The device records how the user interacts with advertisements and information.

[0538] For example, if a user taps on a push notification to view more information, that action is recorded.

[0539] Step 14:

[0540] The terminal transmits the recorded interaction data to the server.

[0541] The server receives the interaction data and stores it in a database.

[0542] Step 15:

[0543] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[0544] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[0545] Through these steps, users will be provided with personalized advertisements and local information in real time, creating a system that will contribute to revitalizing the local economy and promoting tourism.

[0546] Example 1

[0547] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0548] Conventional ad delivery systems are unable to fully utilize users' real-time interests, preferences, location information, and behavioral data, making it difficult to provide truly useful personalized ads and local information to users. Furthermore, there is a lack of technology to effectively analyze this data and provide information at the optimal time. Furthermore, there are few ways to improve the accuracy of subsequent ads and information based on user responses.

[0549] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0550] In this invention, the server includes: means for acquiring user location information; means for transmitting user data, including the acquired location information and interest and preference data, to the server; means for transmitting user behavior data from the terminal to the server; means for the server to analyze regional trends based on the user data and external data using a big data analysis tool; means for inputting the analysis results into a generative AI model to generate personalized advertisements or regional information; means for transmitting the generated advertisements or regional information to the user terminal or display device; means for recording the user's behavior when interacting with the generated advertisements or regional information and transmitting the behavior data to the server; and means for the server to analyze the interaction data and use it to generate the next personalized advertisement or regional information. This makes it possible to effectively utilize the user's real-time location information, interest and preference data, and behavior data to provide personalized information. Furthermore, the accuracy of the service can be further improved based on user feedback.

[0551] "User" refers to an individual or entity that uses the System, specifically the entity that provides location information, interest and preference data, and behavioral data.

[0552] "Location Information" means data that indicates a user's current geographic location, obtained using technologies such as GPS.

[0553] "Interest and Preference Data" refers to information about your interests and preferences that you enter within the App.

[0554] "Behavioral data" refers to data including the history of actions taken by a user within an app, such as search history and browsing history.

[0555] "Server" refers to a computer system that receives, stores, and analyzes data sent by users, and generates and distributes advertisements and local information.

[0556] "Big data analytics tools" are software and technologies that allow for the efficient analysis of vast amounts of data, specifically those used to identify regional trends and demand.

[0557] "Generative AI model" refers to artificial intelligence technology that uses pre-trained algorithms to generate personalized ads and local information based on prompts.

[0558] A "prompt" is an instruction entered into a generative AI model, containing a specific request to generate a particular advertisement or local information.

[0559] "User terminal" refers to a device such as a smartphone or tablet used by a user, which obtains location information and receives advertisements.

[0560] "Display device" refers to a device such as an electronic signboard or display for displaying information from the system.

[0561] "Interaction Data" is data that indicates how a user interacts with an advertisement or local information and is sent to a server.

[0562] MODE FOR CARRYING OUT THE INVENTION

[0563] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. The system functions through a server, terminals, and records of user interactions.

[0564] Retrieving User Information

[0565] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The user can also input their interests and preferences within the app, and this interest and preference data is also sent to the server. Furthermore, user behavior data, such as search history and browsing history within the app, is also collected and sent to the server.

[0566] Big Data Analysis

[0567] The server uses big data analysis tools (e.g., Apache Hadoop) to identify local trends and demand based on the received user data. Analysis includes social media mention counts and foot traffic data. The results of this analysis are fed into a generative AI model (e.g., OpenAI GPT-4).

[0568] Ad generation by generative AI

[0569] The server creates a prompt based on the analysis results and inputs it into the generative AI model. For example, a prompt such as "Please generate an advertisement for a popular art event in Kyoto" is input into the generative AI model. The generative AI model generates personalized advertisements and local information based on this prompt.

[0570] Real-time information distribution

[0571] The generated advertisements and local information are sent from the server to the user's smartphone via push notifications. Using communication technologies such as Firebase Cloud Messaging, the information is delivered to the user instantly. The information is also displayed on display devices (e.g., electronic billboards) when the user passes nearby.

[0572] Recording user interactions

[0573] When a user interacts with an advertisement or information, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is generated.

[0574] Specific examples

[0575] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, the app uses GPS to obtain their current location (Kyoto) and sends it to the server. The server then analyzes social media data and people flow data to determine that the "Kyoto Art Festival" is popular. The server then inputs a prompt message, "Please generate an advertisement for the Kyoto Art Festival," into the generative AI model, which then generates an advertisement for the festival. The generated advertisement is sent from the server to the user's smartphone via push notification, allowing the user to receive the information immediately. In addition, similar information is displayed in real time on nearby electronic billboards. When the user taps the push notification to view more information, their behavioral data is sent to the server and used to generate future advertisements.

[0576] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[0577] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0578] Step 1:

[0579] Retrieving User Information

[0580] When a user launches a smartphone app, the device uses GPS to obtain current location information. This location information is then sent from the device to a server. Specifically, the device receives input data from the GPS and sends it to the server in real time. The output is location data.

[0581] Step 2:

[0582] Enter your interests and preferences

[0583] The user enters their interests and preferences within the app. This data is also sent to the server. Specifically, the device receives the user's input as text data and sends it to the server. The output is interest and preference data.

[0584] Step 3:

[0585] Behavioral data collection

[0586] Every time a user searches or browses within the app, the device records the behavioral data and sends it to the server. Specifically, the device captures the user's action events (search history, browsing history, etc.) and sends them to the server. The output is behavioral data.

[0587] Step 4:

[0588] Data collection

[0589] The server receives location information, interest and preference data, and behavioral data. Specifically, the server acquires this data from each device and stores it in a database. The input is the data sent from the device, and the output is organized user data.

[0590] Step 5:

[0591] Big Data Analysis

[0592] The server uses big data analysis tools (e.g., Apache Hadoop) to identify regional trends and demand. The server analyzes data such as the number of mentions on social media and foot traffic data to extract trend information for each region. The input is user data and external data, and the output is the analysis results.

[0593] Step 6:

[0594] Generate prompt statement

[0595] The server creates a prompt based on the analysis results. Specifically, it generates a text-based instruction to be input into the generative AI model. For example, a prompt might be generated saying, "Please generate an advertisement for a popular art event in Kyoto." The input is the analysis results, and the output is the prompt.

[0596] Step 7:

[0597] AI-powered ad generation

[0598] The prompt text generated by the server is input into a generative AI model (e.g., OpenAI GPT-4) to generate personalized advertisements and local information. Specifically, the AI ​​analyzes the prompt text and generates advertisements and information based on it. The input is the prompt text, and the output is the generated advertisements and local information.

[0599] Step 8:

[0600] Real-time information distribution

[0601] The server sends the generated ads and location information to the user's smartphone via push notification. The information is delivered in real time using Firebase Cloud Messaging. The input is the generated ads and location information, and the output is the push notification.

[0602] Step 9:

[0603] Electronic bulletin board update

[0604] The server sends information to a nearby display device, which displays relevant advertisements and information as the user passes by. The input is the generated advertisement or information, and the output is the display on the display device.

[0605] Step 10:

[0606] Recording user interactions

[0607] Every time a user taps on a push notification to view more information, the device records that action and sends it to the server. The input is the user interaction, and the output is the recorded behavior data.

[0608] Step 11:

[0609] Utilizing data to generate the next ad

[0610] The server analyzes the interaction data and uses it to generate the next personalized advertisement and local information. Specifically, it generates more accurate prompts and advertisements based on past user behavior data. The input is the interaction data, and the output is the analysis results that will be used to generate the next advertisement.

[0611] (Application example 1)

[0612] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0613] While modern autonomous vehicles have advanced in technology to improve safety and efficiency, they still face limitations in providing personalized information and entertainment to passengers. In particular, in tourism and commercial use, there is a lack of ways for passengers to receive information tailored to their interests and preferences in real time to enhance their experience. There is also a need for a mechanism to utilize user feedback on the information provided to provide more highly personalized services.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0615] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information to the server, means for the server to analyze regional trends based on the user data, means for inputting the analysis results into a generative AI model to generate personalized tourist spot information or advertisements, means for transmitting the generated tourist spot information or advertisements to a user terminal or display device in the vehicle, means for recording user behavior when interacting with the generated tourist spot information or advertisements and transmitting the behavioral data to the server, and means for the server to analyze the interaction data and use it to generate the next personalized tourist spot information or advertisements. This allows passengers to receive personalized information in real time while in an autonomous vehicle, improving their experience and continuously improving the accuracy of the information provided.

[0616] An "autonomous vehicle" is a vehicle that does not require a driver and operates using an automatic control system.

[0617] "Location Information" means data that indicates a specific location using GPS or other location measurement technology.

[0618] "User Data" refers to data that includes general information about a user, such as the user's location, interests, preferences, and behavioral data.

[0619] "Server" means a computer system that collects, analyzes, processes data, and distributes the generated information.

[0620] "Regional trends" refers to information about events, places, and other things that are currently popular or attracting attention in a particular region.

[0621] "Analysis" is the process of extracting useful information and patterns from collected data.

[0622] A "generative AI model" is an algorithm that uses artificial intelligence to generate personalized information from input data.

[0623] "Tourist spot information" refers to detailed information about tourist spots.

[0624] An "advertisement" is an informational message created to promote a particular product, service, etc.

[0625] A "user device" is a device that can be directly operated by a user, such as a smartphone or tablet.

[0626] A "display device" is a display or screen used to visually present information to a user.

[0627] "Interact" means that a user interacts in some way with the information or application provided.

[0628] "Behavioral Data" means data that describes how a user interacts with an application or the information provided to them.

[0629] This invention aims to realize a system for providing personalized information to passengers in an autonomous vehicle. Specifically, the system is constructed by combining the following steps.

[0630] First, the server uses GPS to obtain the location information of the autonomous vehicle. The GPS module is installed in the vehicle and periodically determines its current location. This location information is sent to the server.

[0631] The server then obtains passengers' interests and preferences through user devices such as tablets and smartphones. Passengers input their interests and preferences through tablets installed in the vehicle or through an app on their own smartphone. This data is also sent to the server and stored as user data.

[0632] The server also collects behavioral data, such as user search and browsing history within the app. Based on this data, the server analyzes trends and demand by region. A big data analysis system (e.g., Apache Spark) is used for the analysis, and the popularity and key points of events in a specific region are identified based on the number of mentions on social media.

[0633] The server then inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate personalized tourist spot information and advertisements. An example of a specific prompt might be, "The user is interested in historical buildings. Please tell me about popular historical buildings as the latest tourist attractions near their current location."

[0634] The generated tourist information and advertisements are sent in real time to tablets and smartphones on board, giving passengers instant access to personalized information.

[0635] Finally, when a passenger interacts with the displayed information (e.g., tapping on a coupon), their actions are again sent to the server, which analyzes this interaction data and uses it the next time the information is provided, further improving the accuracy and personalization of the information provided.

[0636] For example, if a passenger expresses an interest in "historical buildings," the server analyzes local trends based on the acquired location information and generates information and coupons related to "historical castles," a popular tourist spot nearby. This information is displayed in real time on the tablet, and when the passenger uses the coupon, the data is sent back to the server to help personalize the service for future visits.

[0637] In this way, the invention can improve the passenger experience in autonomous vehicles and continuously improve the accuracy of the information provided.

[0638] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0639] Step 1:

[0640] (Processing content) Acquisition and transmission of location information

[0641] (Specific operation) The GPS module installed in the autonomous vehicle periodically acquires its current location. This location information is sent to a server via a communication module in the vehicle.

[0642] (Input) Current position from the GPS module

[0643] (Data processing) Format conversion of location information

[0644] (Output) Location data sent to the server

[0645] Step 2:

[0646] (Processing content) Acquiring user interests and preferences

[0647] (Specific operation) The user inputs their interests and preferences using a tablet installed in the vehicle or an app on the passenger's smartphone. The input data is sent from the app to the server.

[0648] (Input) Interest and preference data from tablet and smartphone apps

[0649] (Data processing) Linking user ID with interest and preference data

[0650] (Output) User interest and preference data sent to the server

[0651] Step 3:

[0652] (Processing content) Collection of user behavior data

[0653] (Specific operation) Collects behavioral data such as search history and browsing history within the app and sends it to the server. To do this, the app generates log data and periodically sends it to the server.

[0654] (Input) Search history and browsing history within the app

[0655] (Data processing) Standardizing the format of behavioral data

[0656] (Output) Action log data sent to the server

[0657] Step 4:

[0658] (Processing content)Analysis of regional trends

[0659] (Specific operation) The server uses the collected user data and location information to analyze regional trends using a big data analysis system (e.g., Apache Spark). Trends are identified based on data such as the number of mentions on social media and the flow of people.

[0660] (Input) User data, location information

[0661] (Data processing) Big data aggregation and statistical analysis

[0662] (Output) Regional trend data as analysis results

[0663] Step 5:

[0664] (Processing content) Creation of personalized information

[0665] (Specific operation) The server inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate tourist spot information and advertisements appropriate for the user. A prompt sentence is input into the generative AI model, and it is generated based on this.

[0666] (Input) Regional trend data, prompt statement

[0667] (Data processing) Information generation using generative AI models

[0668] (Output) Personalized information (tourist spot information, advertisements)

[0669] Step 6:

[0670] (Processing content) Real-time information distribution

[0671] (Specific operation) The generated personalized information is sent to tablets and smartphones in the vehicle and notifications are sent, allowing passengers to receive information in real time.

[0672] (Input) Personalization Information

[0673] (Data processing) Creation and sending of notification messages

[0674] (Output) Information displayed on the user's terminal

[0675] Step 7:

[0676] (Processing content) Recording of user interactions

[0677] (Specific actions) When the user interacts with the displayed information (tap, click, use, etc.), that action is sent back to the server, which stores this data and uses it for the next analysis.

[0678] (Input) User interaction data

[0679] (Data processing) Recording interaction data and integrating it into analytical data

[0680] (Output) Interaction data stored on the server

[0681] Step 8:

[0682] (Processing content) Analysis for the next information provision

[0683] (Specific operation) The server analyzes the interaction data and uses it to generate personalized information for the next visit. This analysis makes it possible to provide more accurate information.

[0684] (Input) Interaction data

[0685] (Data processing) Modify the model for the next generation of personalized information

[0686] (Output) Analysis data for the next information provision

[0687] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0688] MODE FOR CARRYING OUT THE INVENTION

[0689] This invention is a system that provides personalized advertisements and local information in real time using an emotion engine that recognizes user emotions in addition to user location information, interests, preferences, and behavioral data. This system functions through a server, terminals, the emotion engine, and records of user interactions.

[0690] Retrieving User Information

[0691] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[0692] Acquiring emotion data

[0693] The device is equipped with an emotion engine that analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data, which is then sent to a server in real time and stored as the user's emotional profile.

[0694] Big Data Analysis

[0695] The server analyzes the collected user data and sentiment data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[0696] Ad generation by generative AI

[0697] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history and current emotional state.

[0698] Real-time information distribution

[0699] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by. In addition, the timing of information delivery can be adjusted based on the user's emotions recognized by the emotion engine. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[0700] Recording user interactions

[0701] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided. Emotional data evoked by the user's interaction is also recorded and sent to the server as feedback.

[0702] Specific examples

[0703] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The emotion engine then recognizes the user's relaxed emotions. Based on this information, the generative AI creates an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, based on the user's past usage history and current emotional state, coupons for nearby restaurants are also generated and provided at the same time. The user receives this information, and the data on their interactions is sent back to the server to be used for providing information in the future.

[0704] In this way, the system can provide users with useful information in real time, contributing to regional economic revitalization and tourism promotion. It can also continuously collect user feedback to improve the accuracy of the information provided. The introduction of an emotion engine enables more advanced personalization based on the user's emotional state, further improving the effectiveness of information provision.

[0705] The processing flow will be explained below.

[0706] Specific processing steps of the program

[0707] Information Acquisition Phase

[0708] Step 1:

[0709] The user launches the smartphone app and logs in.

[0710] The device uses GPS to obtain the user's current location.

[0711] Step 2:

[0712] The location information acquired by the device is sent to the server.

[0713] The server receives the location information and determines the user's location.

[0714] Step 3:

[0715] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[0716] The device sends this selection data to the server.

[0717] Step 4:

[0718] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[0719] The server receives the behavioral data and stores it in a database.

[0720] Emotion data acquisition phase

[0721] Step 5:

[0722] The device uses a camera and microphone to detect the user's facial expressions and tone of voice.

[0723] The emotion engine analyzes this data to recognize the user's emotions.

[0724] Step 6:

[0725] The recognized emotion data is sent to the server in real time.

[0726] The server receives the emotion data and adds it to the user profile.

[0727] Data analysis phase

[0728] Step 7:

[0729] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[0730] The server also collects social media data and local pedestrian flow data.

[0731] Step 8:

[0732] The server cleanses the data and formats it into an easy-to-parse format.

[0733] Incomplete or incorrect data is filtered out.

[0734] Step 9:

[0735] The server analyzes regional trends.

[0736] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[0737] Ad generation phase

[0738] Step 10:

[0739] The server inputs the analysis results into the generative model.

[0740] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, emotional data, and trend data.

[0741] Step 11:

[0742] The server utilizes the generative model to generate personalized advertisements or local information.

[0743] For example, if a user is near a museum, ads for art exhibitions will be generated.

[0744] Step 12:

[0745] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[0746] The server finalizes the advertisement and information.

[0747] Information distribution phase

[0748] Step 13:

[0749] The server sends the generated advertisements and local information to the user's terminal.

[0750] The device will display information to the user via push notifications or at specific times.

[0751] Step 14:

[0752] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[0753] Devices and electronic billboards display information relevant to the user's current location.

[0754] Step 15:

[0755] Based on the emotional data, the server adjusts the timing of information distribution.

[0756] For example, it could deliver restaurant promotion information when the user is relaxing.

[0757] Interaction Recording Phase

[0758] Step 16:

[0759] The device records how the user interacts with advertisements and information.

[0760] For example, if a user taps on a push notification to view more information, that action is recorded.

[0761] Step 17:

[0762] The terminal transmits the recorded interaction data to the server.

[0763] The server receives the interaction data and stores it in a database.

[0764] Step 18:

[0765] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[0766] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[0767] Through these steps, a system will be built that provides users with personalized advertisements and local information in real time, contributing to the revitalization of local economies and the promotion of tourism. Furthermore, the introduction of an emotion engine will provide more appropriate information based on the user's emotional state, further improving the effectiveness of information provision.

[0768] Example 2

[0769] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0770] Conventional personalized advertising and local information provision systems typically generate advertisements and information based on users' location information, interests, preferences, and behavioral data. However, personalization that takes into account the user's emotional state is not adequate, and the timing and content are often inappropriate. As a result, there are issues with not being able to attract users' attention and reducing the effectiveness of advertisements and information.

[0771] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0772] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information, interests, preferences, and behavioral data to the server, means for acquiring user emotion data using an emotion engine installed in the terminal, means for transmitting the acquired emotion data to the server, means for the server to analyze regional trends based on the user data and emotion data and generate a prompt message using a generative AI model, means for generating personalized advertisements or regional information based on the generated prompt message, means for transmitting the generated advertisements or regional information to the user terminal or electronic billboard, and means for adjusting the timing of information delivery based on the user's emotional state, thereby enabling the provision of more effective personalized advertisements and regional information according to the user's emotional state.

[0773] "Location information" is geographic coordinate data that indicates where a user is currently located.

[0774] "Interests and Preferences" refers to information about a user's preferred activities, places, items, etc.

[0775] "Behavioral data" refers to information such as search history, browsing history, and click history when a user uses the system.

[0776] "Emotional data" refers to information about a user's emotional state, analyzed from facial expressions, tone of voice, etc.

[0777] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[0778] A "server" is a remote computer system that receives and analyzes user data and emotional data.

[0779] A "generative AI model" is a machine learning model that generates prompts based on user data and emotional data to create personalized advertisements and information.

[0780] A "prompt sentence" is an input sentence that a generative AI model uses to generate personalized ads or information.

[0781] An "advertisement" is text, image, or video that provides commercial information to users.

[0782] "Local information" is information about events and places associated with a particular geographic area.

[0783] A "push notification" is a notification message that provides information to a user's device in real time.

[0784] An "electronic billboard" is an electronic display device for displaying advertisements and information in a specific area.

[0785] "Interact" refers to the user's reaction to and manipulation of the advertisements and information provided.

[0786] "Interaction data" is a record of the actions and behaviors that users take in response to advertisements or information.

[0787] "Feedback" is information used to improve the system's personalization accuracy based on user emotional and interaction data.

[0788] This invention is a system that uses user location information, interests, preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time. This system is mainly implemented using a server, a terminal, an emotion engine, and a generative AI model. Detailed embodiments of the system are described below.

[0789] Retrieving User Information

[0790] When a user launches the smartphone app, the device uses GPS to obtain its current location information, which is then captured as latitude and longitude data and sent to the server in JSON format.

[0791] Next, users enter their interests and preferences within the app. For example, they enter information such as their interest in museums and restaurants in text format, and this data is also sent to the server. In addition, the device collects behavioral data such as search history, browsing history, and click history within the app, and this information is also sent to the server in JSON format.

[0792] Acquiring emotion data

[0793] The device is equipped with an emotion engine that captures the user's emotional data. The emotion engine uses a camera to capture the user's facial expressions and analyzes their emotional state using a facial recognition algorithm. It also uses a microphone to capture the tone of voice and analyzes it using a voice recognition algorithm. The captured emotion data is sent to a server in real time and stored as the user's emotional profile.

[0794] Big Data Analysis

[0795] The server integrates the acquired user data and sentiment data to analyze trends and demand in each region. Specifically, it analyzes big data including the number of mentions on social media and population flow data. This analysis can, for example, identify the popularity of events in a specific region and identify new points of interest.

[0796] Ad generation by generative AI

[0797] The server inputs the analysis results into a generative AI model to generate personalized advertisements and local information. For example, if a user is interested in museums, the generative AI model will generate advertisements based on the following prompt:

[0798] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[0799] The generative AI model uses this prompt to create the most relevant ads and local information for the user.

[0800] Real-time information distribution

[0801] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access the information instantly. On electronic billboards, relevant information is displayed in real time as the user passes by. The emotion engine also adjusts the timing of information delivery based on the user's emotional state. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[0802] Recording user interactions

[0803] When a user interacts with an advertisement or information provided through an app or electronic billboard, that behavior is recorded and sent to a server. For example, if a user taps on a push notification to view more information, that behavioral data is sent to the server. This interaction data is used to provide advertisements and information in the future, enabling more accurate personalization.

[0804] In this way, the system integrates users' location information, interests and preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time, enabling it to accurately deliver useful information to users.

[0805] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0806] Step 1:

[0807] Retrieving User Information

[0808] The device uses GPS to obtain the user's current location information. The input data is the latitude and longitude received from the GPS sensor. The output data is the obtained location information (latitude and longitude) converted to JSON format.

[0809] The device sends this location information to the server. Specifically, the device sends the location information to the server using an HTTP POST request.

[0810] Step 2:

[0811] Enter your interests and preferences

[0812] Users input their interests and preferences into a smartphone app. The input data is information about interests and preferences that users enter by tapping or entering text.

[0813] The device converts this information into JSON format and sends it to the server. The output data is JSON format data of the user's interests and preferences. Specifically, the device collects data entered through the user interface and sends it to the server.

[0814] Step 3:

[0815] Behavioral data collection

[0816] The device collects the user's search history, browsing history, and click history within the app. The input data is a log of the user's actions.

[0817] The device converts this data into JSON format and sends it to the server. The output data is JSON formatted behavioral data. Specifically, it monitors the action log and sends the data to the server in real time. As with other data, it uses an HTTP POST request.

[0818] Step 4:

[0819] Acquiring emotion data

[0820] The emotion engine installed on the device analyzes the user's facial expressions. The input data is image data of the user's face captured by the camera. The output data is data indicating the analyzed emotional state (e.g., smiling, angry, surprised, etc.).

[0821] The device sends this data to a server, which uses a facial recognition algorithm to analyze the user's emotional state and sends the results to the server via an HTTP POST request.

[0822] Step 5:

[0823] Voice Tone Analysis

[0824] The device analyzes the tone of the user's voice. The input data is the voice data captured by the microphone. The output data is data indicating the analyzed emotional state (e.g., stressed state, relaxed state, etc.).

[0825] The device sends this data to the server, which uses a speech recognition algorithm to analyze the tone of the voice and sends the results to the server via an HTTP POST request.

[0826] Step 6:

[0827] Big Data Analysis

[0828] The server integrates the received user data and sentiment data to analyze local trends and demand. The input data is the collected user data and sentiment data. Additional big data (e.g., social media mention counts and population mobility data) is also used for analysis.

[0829] The server analyzes this data to identify trends and demand in specific regions. The output data is data showing trends and demand by region. Specifically, it runs big data analysis algorithms to identify trends by region.

[0830] Step 7:

[0831] Ad generation by generative AI

[0832] The server generates prompt sentences for the generative AI model based on the results of big data analysis. The input data is the analysis results, the user's interests and preferences, behavioral data, and emotional data. Examples of prompt sentences are:

[0833] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[0834] The generative AI model generates personalized advertisements and local information based on this prompt. The output data is personalized advertisements and local information. Specifically, the generative AI model analyzes the prompt and generates appropriate advertisements and information.

[0835] Step 8:

[0836] Real-time information distribution

[0837] The server transmits the generated advertisement and area information to the user terminal or the electronic billboard, and the input data is the generated advertisement and area information.

[0838] The server delivers this data in real time, providing information to users via push notifications on smartphones and real-time displays on electronic billboards. Specifically, the server sends the generated advertisements and local information to the delivery system, triggering push notifications.

[0839] Step 9:

[0840] Recording user interactions

[0841] When a user interacts with the advertisements or information provided, their actions are recorded. The input data is a log of the user's interaction (for example, tapping on a push notification to view more information).

[0842] The device sends this interaction data to the server. The output data is an interaction log in JSON format. Specifically, the device detects interaction events and sends the data to the server in real time.

[0843] Step 10:

[0844] Data Feedback

[0845] The server analyzes the user's interaction data and uses it to generate personalized ads and local information for the next visit. The input data is the collected interaction data.

[0846] The server analyzes this data to identify emerging trends and patterns. The output is an optimization profile for the next ad or information offering. Specifically, machine learning algorithms are used to analyze the interaction data and use the results to improve the accuracy of future ads and information offerings. Emotional feedback is also included in the analysis.

[0847] (Application example 2)

[0848] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0849] Conventional advertising systems have a low level of personalization accuracy, making it difficult to provide users with timely and useful information. Furthermore, delivering ads uniformly without considering the user's current emotional state can be counterproductive. For example, if an ad is sent to a user when they are tired, it may not only not pique their interest, but may also create a negative impression of the ad itself. Furthermore, there is no established method for effectively utilizing real-time location information and emotional data, making it difficult to provide information tailored to local trends and users' interests and preferences.

[0850] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for transmitting user data and emotion data to the server, means for analyzing regional trends based on the user data and emotion data, and means for inputting the analysis results into a generative AI model and generating personalized advertisements or regional information based on the user's current location and emotional state. This makes it possible to provide timely and highly accurate personalized information that takes into account the user's current location and emotional state.

[0851] "User Data" is a general term for all data related to a user, such as the user's location information, interests and preferences, and behavioral history.

[0852] "Emotional data" is information about a user's emotional state at any given time, obtained by analyzing their facial expressions, tone of voice, and other biometric signals.

[0853] A "server" is a computer system that processes various data, such as analyzing user data and emotional data, generating advertisements, and providing information.

[0854] "Local trends" refers to information that shows trends and patterns regarding user interests, popular events, facilities, etc. within a particular geographic area.

[0855] "Generative AI models" refer to artificial intelligence algorithms and learning models used to generate personalized ads and location information using user and emotional data as input.

[0856] "Personalized advertising or location information" refers to advertising or location information that is individually optimized based on a user's individual interests, preferences, behavioral history, and emotional state.

[0857] "User terminal" refers to a communication device used by a user, such as a smartphone or tablet.

[0858] "Display device" refers to a device, such as an electronic bulletin board or digital signage, that visually presents advertisements or information to users.

[0859] "Push notification" is a messaging service that sends information from a server to a user's device in real time.

[0860] "Interact" means that a user interacts in some way with the advertisement or location information provided.

[0861] This invention is a system that collects and analyzes user location information, interests, preferences, behavioral data, and emotional data, and provides personalized advertisements and local information in real time. This system operates mainly using the following hardware and software.

[0862] Hardware

[0863] 1. User device: a smartphone or tablet, including a GPS and a camera.

[0864] 2. Display devices: digital signage and electronic bulletin boards.

[0865] software

[0866] 1. Emotion Engine: Facial expression recognition and voice analysis software to capture the user's emotional state.

[0867] 2. Generative AI model: Runs on the server and generates personalized ads and local information based on user data and emotional data.

[0868] 3. Data transmission / reception module: A communication module for exchanging data between the user terminal and the server.

[0869] Process Overview

[0870] 1. Acquisition and transmission of user data

[0871] When a user launches a smartphone app, the device's GPS function is used to obtain current location information. The app also collects the user's interests, preferences, and behavioral history. This data is sent to a server in real time.

[0872] 2. Acquiring Emotion Data

[0873] The smartphone is equipped with an emotion engine that analyzes the user's facial expressions and tone of voice, and this acquires emotional data, which is also sent to the server.

[0874] 3. Data analysis and ad generation

[0875] On the server side, user and sentiment data is analyzed to understand specific local trends and user interests, which is then used by generative AI models to generate personalized ads and local information.

[0876] 4. Provision of Information

[0877] The generated advertisements and local information are sent in real time to the user's device or display device, and on smartphones, the information is provided via push notifications.

[0878] Specific examples

[0879] For example, if a user is in a shopping mall, the generative AI model can generate special sale information and coupons for the mall based on the smartphone's location information, collected interest and preference data, and the relaxed state detected by the emotion engine, and provide them to the user via push notification.

[0880] Prompt Sentence Examples

[0881] "Use up-to-date sentiment data to generate ads that are best suited to relaxed users in a specific location."

[0882] This will enable us to provide users with the information they need in a timely manner, improving the effectiveness of our advertising. In addition, by collecting user feedback data, we will be able to further improve the accuracy of the information we provide next time.

[0883] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0884] Step 1:

[0885] When a user launches a smartphone app, the device uses its GPS function to obtain the user's current location. This location information is input data that the application sends to the server as user data. This location information is used to inform the server of the user's current location. Specifically, the smartphone's GPS sensor obtains the location information, and the application sends it to the server via an HTTP request.

[0886] Step 2:

[0887] The user's interests, preferences, and past behavioral data are also collected at the same time. This information, such as the user's past search keywords and visited places, is sent to the server along with the interest and preference data entered within the app. This input data forms an individual user profile and is used by the server to analyze the user's interests. Specifically, past history data is collected from the application's database and sent to the server along with location information.

[0888] Step 3:

[0889] The device is equipped with an emotion engine that analyzes the user's facial expressions (using the camera) and voice tone (using the microphone) to obtain emotion data. This emotion data is also sent to the server. The emotion data as input data indicates the user's current emotional state and is used to personalize advertisements. Specifically, the device uses the smartphone's camera and microphone to collect the user's biometric data, and the results analyzed by the emotion engine are sent to the server.

[0890] Step 4:

[0891] The server analyzes the received location information, interest and preference data, behavioral data, and emotional data. For this analysis, it uses big data analytics technology to gain insights based on local trends and user interests. This process includes data processing and calculation, such as data retrieval from the database, cleansing, point analysis, and trend analysis. Specifically, it retrieves trend data from the database and extracts information on popular events and stores by region.

[0892] Step 5:

[0893] The server inputs the analysis results into a generative AI model to generate personalized ads or local information based on the user's current location and emotional state. In this generation process, the generative AI model generates the optimal ads or information based on the input data and generates the results. Specifically, the generative AI model matches the user's profile with trend data to generate the optimal ads or information. An example of a prompt is, "Use the latest emotional data to generate the optimal ads for relaxed users in a specific area."

[0894] Step 6:

[0895] The generated advertisements and local information are sent from the server to the user's device or display device. In the case of smartphones, the information is provided to the user via push notification. The generated results as input data are sent to the device as an HTTP response, and the device displays the push notification. Specifically, the server returns the generated results as an HTTP response, and the smartphone receives the data and triggers the push notification.

[0896] Step 7:

[0897] When a user interacts with the generated advertisement or local information, the behavioral data is recorded and sent to the server. This behavioral data includes the click history of push notifications and the time spent viewing advertisements. Specifically, the system captures user operation events and sends them to the server as behavioral data.

[0898] Step 8:

[0899] The server analyzes the interaction data and uses it to generate the next personalized advertisements and local information. This analysis process includes analyzing past behavioral patterns and evaluating emotional data. Specifically, the behavioral history stored in the database is analyzed and used as input data for the next generation AI model.

[0900] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0901] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0902] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0903] [Third embodiment]

[0904] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0905] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0906] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0907] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0908] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0909] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0910] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0911] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0912] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0913] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0914] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0915] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0916] MODE FOR CARRYING OUT THE INVENTION

[0917] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. This system works through a server, terminals, and records of user interactions.

[0918] Retrieving User Information

[0919] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[0920] Big Data Analysis

[0921] The server analyzes the collected user data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[0922] Ad generation by generative AI

[0923] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history.

[0924] Real-time information distribution

[0925] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by.

[0926] Recording user interactions

[0927] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided.

[0928] Specific examples

[0929] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The generation AI uses this information to create an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, coupons for nearby restaurants are also generated and provided at the same time based on the user's past usage history. The user receives this information, and the data on their interactions is sent back to the server to be used to provide information in the future.

[0930] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[0931] The processing flow will be explained below.

[0932] Specific processing steps of the program

[0933] Information Acquisition Phase

[0934] Step 1:

[0935] The user launches the smartphone app and logs in.

[0936] The device uses GPS to obtain the user's current location.

[0937] Step 2:

[0938] The location information acquired by the device is sent to the server.

[0939] The server receives the location information and determines the user's location.

[0940] Step 3:

[0941] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[0942] The device sends this selection data to the server.

[0943] Step 4:

[0944] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[0945] The server receives the behavioral data and stores it in a database.

[0946] Data analysis phase

[0947] Step 5:

[0948] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[0949] The server also collects social media data and local pedestrian flow data.

[0950] Step 6:

[0951] The server cleanses the data and formats it into an easy-to-parse format.

[0952] Incomplete or incorrect data is filtered out.

[0953] Step 7:

[0954] The server analyzes regional trends.

[0955] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[0956] Ad generation phase

[0957] Step 8:

[0958] The server inputs the analysis results into the generative model.

[0959] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, and trend data.

[0960] Step 9:

[0961] The server utilizes the generative model to generate personalized advertisements or local information.

[0962] For example, if a user is near a museum, ads for art exhibitions will be generated.

[0963] Step 10:

[0964] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[0965] The server finalizes the advertisement and information.

[0966] Information distribution phase

[0967] Step 11:

[0968] The server sends the generated advertisements and local information to the user's terminal.

[0969] The device will display information to the user via push notifications or at specific times.

[0970] Step 12:

[0971] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[0972] Devices and electronic billboards display information relevant to the user's current location.

[0973] Interaction Recording Phase

[0974] Step 13:

[0975] The device records how the user interacts with advertisements and information.

[0976] For example, if a user taps on a push notification to view more information, that action is recorded.

[0977] Step 14:

[0978] The terminal transmits the recorded interaction data to the server.

[0979] The server receives the interaction data and stores it in a database.

[0980] Step 15:

[0981] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[0982] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[0983] Through these steps, users will be provided with personalized advertisements and local information in real time, creating a system that will contribute to revitalizing the local economy and promoting tourism.

[0984] Example 1

[0985] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0986] Conventional ad delivery systems are unable to fully utilize users' real-time interests, preferences, location information, and behavioral data, making it difficult to provide truly useful personalized ads and local information to users. Furthermore, there is a lack of technology to effectively analyze this data and provide information at the optimal time. Furthermore, there are few ways to improve the accuracy of subsequent ads and information based on user responses.

[0987] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0988] In this invention, the server includes: means for acquiring user location information; means for transmitting user data, including the acquired location information and interest and preference data, to the server; means for transmitting user behavior data from the terminal to the server; means for the server to analyze regional trends based on the user data and external data using a big data analysis tool; means for inputting the analysis results into a generative AI model to generate personalized advertisements or regional information; means for transmitting the generated advertisements or regional information to the user terminal or display device; means for recording the user's behavior when interacting with the generated advertisements or regional information and transmitting the behavior data to the server; and means for the server to analyze the interaction data and use it to generate the next personalized advertisement or regional information. This makes it possible to effectively utilize the user's real-time location information, interest and preference data, and behavior data to provide personalized information. Furthermore, the accuracy of the service can be further improved based on user feedback.

[0989] "User" refers to an individual or entity that uses the System, specifically the entity that provides location information, interest and preference data, and behavioral data.

[0990] "Location Information" means data that indicates a user's current geographic location, obtained using technologies such as GPS.

[0991] "Interest and Preference Data" refers to information about your interests and preferences that you enter within the App.

[0992] "Behavioral data" refers to data including the history of actions taken by a user within an app, such as search history and browsing history.

[0993] "Server" refers to a computer system that receives, stores, and analyzes data sent by users, and generates and distributes advertisements and local information.

[0994] "Big data analytics tools" are software and technologies that allow for the efficient analysis of vast amounts of data, specifically those used to identify regional trends and demand.

[0995] "Generative AI model" refers to artificial intelligence technology that uses pre-trained algorithms to generate personalized ads and local information based on prompts.

[0996] A "prompt" is an instruction entered into a generative AI model, containing a specific request to generate a particular advertisement or local information.

[0997] "User terminal" refers to a device such as a smartphone or tablet used by a user, which obtains location information and receives advertisements.

[0998] "Display device" refers to a device such as an electronic signboard or display for displaying information from the system.

[0999] "Interaction Data" is data that indicates how a user interacts with an advertisement or local information and is sent to a server.

[1000] MODE FOR CARRYING OUT THE INVENTION

[1001] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. The system functions through a server, terminals, and records of user interactions.

[1002] Retrieving User Information

[1003] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The user can also input their interests and preferences within the app, and this interest and preference data is also sent to the server. Furthermore, user behavior data, such as search history and browsing history within the app, is also collected and sent to the server.

[1004] Big Data Analysis

[1005] The server uses big data analysis tools (e.g., Apache Hadoop) to identify local trends and demand based on the received user data. Analysis includes social media mention counts and foot traffic data. The results of this analysis are fed into a generative AI model (e.g., OpenAI GPT-4).

[1006] Ad generation by generative AI

[1007] The server creates a prompt based on the analysis results and inputs it into the generative AI model. For example, a prompt such as "Please generate an advertisement for a popular art event in Kyoto" is input into the generative AI model. The generative AI model generates personalized advertisements and local information based on this prompt.

[1008] Real-time information distribution

[1009] The generated advertisements and local information are sent from the server to the user's smartphone via push notifications. Using communication technologies such as Firebase Cloud Messaging, the information is delivered to the user instantly. The information is also displayed on display devices (e.g., electronic billboards) when the user passes nearby.

[1010] Recording user interactions

[1011] When a user interacts with an advertisement or information, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is generated.

[1012] Specific examples

[1013] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, the app uses GPS to obtain their current location (Kyoto) and sends it to the server. The server then analyzes social media data and people flow data to determine that the "Kyoto Art Festival" is popular. The server then inputs a prompt message, "Please generate an advertisement for the Kyoto Art Festival," into the generative AI model, which then generates an advertisement for the festival. The generated advertisement is sent from the server to the user's smartphone via push notification, allowing the user to receive the information immediately. In addition, similar information is displayed in real time on nearby electronic billboards. When the user taps the push notification to view more information, their behavioral data is sent to the server and used to generate future advertisements.

[1014] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[1015] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1016] Step 1:

[1017] Retrieving User Information

[1018] When a user launches a smartphone app, the device uses GPS to obtain current location information. This location information is then sent from the device to a server. Specifically, the device receives input data from the GPS and sends it to the server in real time. The output is location data.

[1019] Step 2:

[1020] Enter your interests and preferences

[1021] The user enters their interests and preferences within the app. This data is also sent to the server. Specifically, the device receives the user's input as text data and sends it to the server. The output is interest and preference data.

[1022] Step 3:

[1023] Behavioral data collection

[1024] Every time a user searches or browses within the app, the device records the behavioral data and sends it to the server. Specifically, the device captures the user's action events (search history, browsing history, etc.) and sends them to the server. The output is behavioral data.

[1025] Step 4:

[1026] Data collection

[1027] The server receives location information, interest and preference data, and behavioral data. Specifically, the server acquires this data from each device and stores it in a database. The input is the data sent from the device, and the output is organized user data.

[1028] Step 5:

[1029] Big Data Analysis

[1030] The server uses big data analysis tools (e.g., Apache Hadoop) to identify regional trends and demand. The server analyzes data such as the number of mentions on social media and foot traffic data to extract trend information for each region. The input is user data and external data, and the output is the analysis results.

[1031] Step 6:

[1032] Generate prompt statement

[1033] The server creates a prompt based on the analysis results. Specifically, it generates a text-based instruction to be input into the generative AI model. For example, a prompt might be generated saying, "Please generate an advertisement for a popular art event in Kyoto." The input is the analysis results, and the output is the prompt.

[1034] Step 7:

[1035] AI-powered ad generation

[1036] The prompt text generated by the server is input into a generative AI model (e.g., OpenAI GPT-4) to generate personalized advertisements and local information. Specifically, the AI ​​analyzes the prompt text and generates advertisements and information based on it. The input is the prompt text, and the output is the generated advertisements and local information.

[1037] Step 8:

[1038] Real-time information distribution

[1039] The server sends the generated ads and location information to the user's smartphone via push notification. The information is delivered in real time using Firebase Cloud Messaging. The input is the generated ads and location information, and the output is the push notification.

[1040] Step 9:

[1041] Electronic bulletin board update

[1042] The server sends information to a nearby display device, which displays relevant advertisements and information as the user passes by. The input is the generated advertisement or information, and the output is the display on the display device.

[1043] Step 10:

[1044] Recording user interactions

[1045] Every time a user taps on a push notification to view more information, the device records that action and sends it to the server. The input is the user interaction, and the output is the recorded behavior data.

[1046] Step 11:

[1047] Utilizing data to generate the next ad

[1048] The server analyzes the interaction data and uses it to generate the next personalized advertisement and local information. Specifically, it generates more accurate prompts and advertisements based on past user behavior data. The input is the interaction data, and the output is the analysis results that will be used to generate the next advertisement.

[1049] (Application example 1)

[1050] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1051] While modern autonomous vehicles have advanced in technology to improve safety and efficiency, they still face limitations in providing personalized information and entertainment to passengers. In particular, in tourism and commercial use, there is a lack of ways for passengers to receive information tailored to their interests and preferences in real time to enhance their experience. There is also a need for a mechanism to utilize user feedback on the information provided to provide more highly personalized services.

[1052] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1053] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information to the server, means for the server to analyze regional trends based on the user data, means for inputting the analysis results into a generative AI model to generate personalized tourist spot information or advertisements, means for transmitting the generated tourist spot information or advertisements to a user terminal or display device in the vehicle, means for recording user behavior when interacting with the generated tourist spot information or advertisements and transmitting the behavioral data to the server, and means for the server to analyze the interaction data and use it to generate the next personalized tourist spot information or advertisements. This allows passengers to receive personalized information in real time while in an autonomous vehicle, improving their experience and continuously improving the accuracy of the information provided.

[1054] An "autonomous vehicle" is a vehicle that does not require a driver and operates using an automatic control system.

[1055] "Location Information" means data that indicates a specific location using GPS or other location measurement technology.

[1056] "User Data" refers to data that includes general information about a user, such as the user's location, interests, preferences, and behavioral data.

[1057] "Server" means a computer system that collects, analyzes, processes data, and distributes the generated information.

[1058] "Regional trends" refers to information about events, places, and other things that are currently popular or attracting attention in a particular region.

[1059] "Analysis" is the process of extracting useful information and patterns from collected data.

[1060] A "generative AI model" is an algorithm that uses artificial intelligence to generate personalized information from input data.

[1061] "Tourist spot information" refers to detailed information about tourist spots.

[1062] An "advertisement" is an informational message created to promote a particular product, service, etc.

[1063] A "user device" is a device that can be directly operated by a user, such as a smartphone or tablet.

[1064] A "display device" is a display or screen used to visually present information to a user.

[1065] "Interact" means that a user interacts in some way with the information or application provided.

[1066] "Behavioral Data" means data that describes how a user interacts with an application or the information provided to them.

[1067] This invention aims to realize a system for providing personalized information to passengers in an autonomous vehicle. Specifically, the system is constructed by combining the following steps.

[1068] First, the server uses GPS to obtain the location information of the autonomous vehicle. The GPS module is installed in the vehicle and periodically determines its current location. This location information is sent to the server.

[1069] The server then obtains passengers' interests and preferences through user devices such as tablets and smartphones. Passengers input their interests and preferences through tablets installed in the vehicle or through an app on their own smartphone. This data is also sent to the server and stored as user data.

[1070] The server also collects behavioral data, such as user search and browsing history within the app. Based on this data, the server analyzes trends and demand by region. A big data analysis system (e.g., Apache Spark) is used for the analysis, and the popularity and key points of events in a specific region are identified based on the number of mentions on social media.

[1071] The server then inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate personalized tourist spot information and advertisements. An example of a specific prompt might be, "The user is interested in historical buildings. Please tell me about popular historical buildings as the latest tourist attractions near their current location."

[1072] The generated tourist information and advertisements are sent in real time to tablets and smartphones on board, giving passengers instant access to personalized information.

[1073] Finally, when a passenger interacts with the displayed information (e.g., tapping on a coupon), their actions are again sent to the server, which analyzes this interaction data and uses it the next time the information is provided, further improving the accuracy and personalization of the information provided.

[1074] For example, if a passenger expresses an interest in "historical buildings," the server analyzes local trends based on the acquired location information and generates information and coupons related to "historical castles," a popular tourist spot nearby. This information is displayed in real time on the tablet, and when the passenger uses the coupon, the data is sent back to the server to help personalize the service for future visits.

[1075] In this way, the invention can improve the passenger experience in autonomous vehicles and continuously improve the accuracy of the information provided.

[1076] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1077] Step 1:

[1078] (Processing content) Acquisition and transmission of location information

[1079] (Specific operation) The GPS module installed in the autonomous vehicle periodically acquires its current location. This location information is sent to a server via a communication module in the vehicle.

[1080] (Input) Current position from the GPS module

[1081] (Data processing) Format conversion of location information

[1082] (Output) Location data sent to the server

[1083] Step 2:

[1084] (Processing content) Acquiring user interests and preferences

[1085] (Specific operation) The user inputs their interests and preferences using a tablet installed in the vehicle or an app on the passenger's smartphone. The input data is sent from the app to the server.

[1086] (Input) Interest and preference data from tablet and smartphone apps

[1087] (Data processing) Linking user ID with interest and preference data

[1088] (Output) User interest and preference data sent to the server

[1089] Step 3:

[1090] (Processing content) Collection of user behavior data

[1091] (Specific operation) Collects behavioral data such as search history and browsing history within the app and sends it to the server. To do this, the app generates log data and periodically sends it to the server.

[1092] (Input) Search history and browsing history within the app

[1093] (Data processing) Standardizing the format of behavioral data

[1094] (Output) Action log data sent to the server

[1095] Step 4:

[1096] (Processing content)Analysis of regional trends

[1097] (Specific operation) The server uses the collected user data and location information to analyze regional trends using a big data analysis system (e.g., Apache Spark). Trends are identified based on data such as the number of mentions on social media and the flow of people.

[1098] (Input) User data, location information

[1099] (Data processing) Big data aggregation and statistical analysis

[1100] (Output) Regional trend data as analysis results

[1101] Step 5:

[1102] (Processing content) Creation of personalized information

[1103] (Specific operation) The server inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate tourist spot information and advertisements appropriate for the user. A prompt sentence is input into the generative AI model, and it is generated based on this.

[1104] (Input) Regional trend data, prompt statement

[1105] (Data processing) Information generation using generative AI models

[1106] (Output) Personalized information (tourist spot information, advertisements)

[1107] Step 6:

[1108] (Processing content) Real-time information distribution

[1109] (Specific operation) The generated personalized information is sent to tablets and smartphones in the vehicle and notifications are sent, allowing passengers to receive information in real time.

[1110] (Input) Personalization Information

[1111] (Data processing) Creation and sending of notification messages

[1112] (Output) Information displayed on the user's terminal

[1113] Step 7:

[1114] (Processing content) Recording of user interactions

[1115] (Specific actions) When the user interacts with the displayed information (tap, click, use, etc.), that action is sent back to the server, which stores this data and uses it for the next analysis.

[1116] (Input) User interaction data

[1117] (Data processing) Recording interaction data and integrating it into analytical data

[1118] (Output) Interaction data stored on the server

[1119] Step 8:

[1120] (Processing content) Analysis for the next information provision

[1121] (Specific operation) The server analyzes the interaction data and uses it to generate personalized information for the next visit. This analysis makes it possible to provide more accurate information.

[1122] (Input) Interaction data

[1123] (Data processing) Modify the model for the next generation of personalized information

[1124] (Output) Analysis data for the next information provision

[1125] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1126] MODE FOR CARRYING OUT THE INVENTION

[1127] This invention is a system that provides personalized advertisements and local information in real time using an emotion engine that recognizes user emotions in addition to user location information, interests, preferences, and behavioral data. This system functions through a server, terminals, the emotion engine, and records of user interactions.

[1128] Retrieving User Information

[1129] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[1130] Acquiring emotion data

[1131] The device is equipped with an emotion engine that analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data, which is then sent to a server in real time and stored as the user's emotional profile.

[1132] Big Data Analysis

[1133] The server analyzes the collected user data and sentiment data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[1134] Ad generation by generative AI

[1135] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history and current emotional state.

[1136] Real-time information distribution

[1137] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by. In addition, the timing of information delivery can be adjusted based on the user's emotions recognized by the emotion engine. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[1138] Recording user interactions

[1139] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided. Emotional data evoked by the user's interaction is also recorded and sent to the server as feedback.

[1140] Specific examples

[1141] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The emotion engine then recognizes the user's relaxed emotions. Based on this information, the generative AI creates an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, based on the user's past usage history and current emotional state, coupons for nearby restaurants are also generated and provided at the same time. The user receives this information, and the data on their interactions is sent back to the server to be used for providing information in the future.

[1142] In this way, the system can provide users with useful information in real time, contributing to regional economic revitalization and tourism promotion. It can also continuously collect user feedback to improve the accuracy of the information provided. The introduction of an emotion engine enables more advanced personalization based on the user's emotional state, further improving the effectiveness of information provision.

[1143] The processing flow will be explained below.

[1144] Specific processing steps of the program

[1145] Information Acquisition Phase

[1146] Step 1:

[1147] The user launches the smartphone app and logs in.

[1148] The device uses GPS to obtain the user's current location.

[1149] Step 2:

[1150] The location information acquired by the device is sent to the server.

[1151] The server receives the location information and determines the user's location.

[1152] Step 3:

[1153] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[1154] The device sends this selection data to the server.

[1155] Step 4:

[1156] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[1157] The server receives the behavioral data and stores it in a database.

[1158] Emotion data acquisition phase

[1159] Step 5:

[1160] The device uses a camera and microphone to detect the user's facial expressions and tone of voice.

[1161] The emotion engine analyzes this data to recognize the user's emotions.

[1162] Step 6:

[1163] The recognized emotion data is sent to the server in real time.

[1164] The server receives the emotion data and adds it to the user profile.

[1165] Data analysis phase

[1166] Step 7:

[1167] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[1168] The server also collects social media data and local pedestrian flow data.

[1169] Step 8:

[1170] The server cleanses the data and formats it into an easy-to-parse format.

[1171] Incomplete or incorrect data is filtered out.

[1172] Step 9:

[1173] The server analyzes regional trends.

[1174] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[1175] Ad generation phase

[1176] Step 10:

[1177] The server inputs the analysis results into the generative model.

[1178] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, emotional data, and trend data.

[1179] Step 11:

[1180] The server utilizes the generative model to generate personalized advertisements or local information.

[1181] For example, if a user is near a museum, ads for art exhibitions will be generated.

[1182] Step 12:

[1183] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[1184] The server finalizes the advertisement and information.

[1185] Information distribution phase

[1186] Step 13:

[1187] The server sends the generated advertisements and local information to the user's terminal.

[1188] The device will display information to the user via push notifications or at specific times.

[1189] Step 14:

[1190] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[1191] Devices and electronic billboards display information relevant to the user's current location.

[1192] Step 15:

[1193] Based on the emotional data, the server adjusts the timing of information distribution.

[1194] For example, it could deliver restaurant promotion information when the user is relaxing.

[1195] Interaction Recording Phase

[1196] Step 16:

[1197] The device records how the user interacts with advertisements and information.

[1198] For example, if a user taps on a push notification to view more information, that action is recorded.

[1199] Step 17:

[1200] The terminal transmits the recorded interaction data to the server.

[1201] The server receives the interaction data and stores it in a database.

[1202] Step 18:

[1203] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[1204] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[1205] Through these steps, a system will be built that provides users with personalized advertisements and local information in real time, contributing to the revitalization of local economies and the promotion of tourism. Furthermore, the introduction of an emotion engine will provide more appropriate information based on the user's emotional state, further improving the effectiveness of information provision.

[1206] Example 2

[1207] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1208] Conventional personalized advertising and local information provision systems typically generate advertisements and information based on users' location information, interests, preferences, and behavioral data. However, personalization that takes into account the user's emotional state is not adequate, and the timing and content are often inappropriate. As a result, there are issues with not being able to attract users' attention and reducing the effectiveness of advertisements and information.

[1209] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1210] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information, interests, preferences, and behavioral data to the server, means for acquiring user emotion data using an emotion engine installed in the terminal, means for transmitting the acquired emotion data to the server, means for the server to analyze regional trends based on the user data and emotion data and generate a prompt message using a generative AI model, means for generating personalized advertisements or regional information based on the generated prompt message, means for transmitting the generated advertisements or regional information to the user terminal or electronic billboard, and means for adjusting the timing of information delivery based on the user's emotional state, thereby enabling the provision of more effective personalized advertisements and regional information according to the user's emotional state.

[1211] "Location information" is geographic coordinate data that indicates where a user is currently located.

[1212] "Interests and Preferences" refers to information about a user's preferred activities, places, items, etc.

[1213] "Behavioral data" refers to information such as search history, browsing history, and click history when a user uses the system.

[1214] "Emotional data" refers to information about a user's emotional state, analyzed from facial expressions, tone of voice, etc.

[1215] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[1216] A "server" is a remote computer system that receives and analyzes user data and emotional data.

[1217] A "generative AI model" is a machine learning model that generates prompts based on user data and emotional data to create personalized advertisements and information.

[1218] A "prompt sentence" is an input sentence that a generative AI model uses to generate personalized ads or information.

[1219] An "advertisement" is text, image, or video that provides commercial information to users.

[1220] "Local information" is information about events and places associated with a particular geographic area.

[1221] A "push notification" is a notification message that provides information to a user's device in real time.

[1222] An "electronic billboard" is an electronic display device for displaying advertisements and information in a specific area.

[1223] "Interact" refers to the user's reaction to and manipulation of the advertisements and information provided.

[1224] "Interaction data" is a record of the actions and behaviors that users take in response to advertisements or information.

[1225] "Feedback" is information used to improve the system's personalization accuracy based on user emotional and interaction data.

[1226] This invention is a system that uses user location information, interests, preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time. This system is mainly implemented using a server, a terminal, an emotion engine, and a generative AI model. Detailed embodiments of the system are described below.

[1227] Retrieving User Information

[1228] When a user launches the smartphone app, the device uses GPS to obtain its current location information, which is then captured as latitude and longitude data and sent to the server in JSON format.

[1229] Next, users enter their interests and preferences within the app. For example, they enter information such as their interest in museums and restaurants in text format, and this data is also sent to the server. In addition, the device collects behavioral data such as search history, browsing history, and click history within the app, and this information is also sent to the server in JSON format.

[1230] Acquiring emotion data

[1231] The device is equipped with an emotion engine that captures the user's emotional data. The emotion engine uses a camera to capture the user's facial expressions and analyzes their emotional state using a facial recognition algorithm. It also uses a microphone to capture the tone of voice and analyzes it using a voice recognition algorithm. The captured emotion data is sent to a server in real time and stored as the user's emotional profile.

[1232] Big Data Analysis

[1233] The server integrates the acquired user data and sentiment data to analyze trends and demand in each region. Specifically, it analyzes big data including the number of mentions on social media and population flow data. This analysis can, for example, identify the popularity of events in a specific region and identify new points of interest.

[1234] Ad generation by generative AI

[1235] The server inputs the analysis results into a generative AI model to generate personalized advertisements and local information. For example, if a user is interested in museums, the generative AI model will generate advertisements based on the following prompt:

[1236] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[1237] The generative AI model uses this prompt to create the most relevant ads and local information for the user.

[1238] Real-time information distribution

[1239] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access the information instantly. On electronic billboards, relevant information is displayed in real time as the user passes by. The emotion engine also adjusts the timing of information delivery based on the user's emotional state. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[1240] Recording user interactions

[1241] When a user interacts with an advertisement or information provided through an app or electronic billboard, that behavior is recorded and sent to a server. For example, if a user taps on a push notification to view more information, that behavioral data is sent to the server. This interaction data is used to provide advertisements and information in the future, enabling more accurate personalization.

[1242] In this way, the system integrates users' location information, interests and preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time, enabling it to accurately deliver useful information to users.

[1243] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1244] Step 1:

[1245] Retrieving User Information

[1246] The device uses GPS to obtain the user's current location information. The input data is the latitude and longitude received from the GPS sensor. The output data is the obtained location information (latitude and longitude) converted to JSON format.

[1247] The device sends this location information to the server. Specifically, the device sends the location information to the server using an HTTP POST request.

[1248] Step 2:

[1249] Enter your interests and preferences

[1250] Users input their interests and preferences into a smartphone app. The input data is information about interests and preferences that users enter by tapping or entering text.

[1251] The device converts this information into JSON format and sends it to the server. The output data is JSON format data of the user's interests and preferences. Specifically, the device collects data entered through the user interface and sends it to the server.

[1252] Step 3:

[1253] Behavioral data collection

[1254] The device collects the user's search history, browsing history, and click history within the app. The input data is a log of the user's actions.

[1255] The device converts this data into JSON format and sends it to the server. The output data is JSON formatted behavioral data. Specifically, it monitors the action log and sends the data to the server in real time. As with other data, it uses an HTTP POST request.

[1256] Step 4:

[1257] Acquiring emotion data

[1258] The emotion engine installed on the device analyzes the user's facial expressions. The input data is image data of the user's face captured by the camera. The output data is data indicating the analyzed emotional state (e.g., smiling, angry, surprised, etc.).

[1259] The device sends this data to a server, which uses a facial recognition algorithm to analyze the user's emotional state and sends the results to the server via an HTTP POST request.

[1260] Step 5:

[1261] Voice Tone Analysis

[1262] The device analyzes the tone of the user's voice. The input data is the voice data captured by the microphone. The output data is data indicating the analyzed emotional state (e.g., stressed state, relaxed state, etc.).

[1263] The device sends this data to the server, which uses a speech recognition algorithm to analyze the tone of the voice and sends the results to the server via an HTTP POST request.

[1264] Step 6:

[1265] Big Data Analysis

[1266] The server integrates the received user data and sentiment data to analyze local trends and demand. The input data is the collected user data and sentiment data. Additional big data (e.g., social media mention counts and population mobility data) is also used for analysis.

[1267] The server analyzes this data to identify trends and demand in specific regions. The output data is data showing trends and demand by region. Specifically, it runs big data analysis algorithms to identify trends by region.

[1268] Step 7:

[1269] Ad generation by generative AI

[1270] The server generates prompt sentences for the generative AI model based on the results of big data analysis. The input data is the analysis results, the user's interests and preferences, behavioral data, and emotional data. Examples of prompt sentences are:

[1271] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[1272] The generative AI model generates personalized advertisements and local information based on this prompt. The output data is personalized advertisements and local information. Specifically, the generative AI model analyzes the prompt and generates appropriate advertisements and information.

[1273] Step 8:

[1274] Real-time information distribution

[1275] The server transmits the generated advertisement and area information to the user terminal or the electronic billboard, and the input data is the generated advertisement and area information.

[1276] The server delivers this data in real time, providing information to users via push notifications on smartphones and real-time displays on electronic billboards. Specifically, the server sends the generated advertisements and local information to the delivery system, triggering push notifications.

[1277] Step 9:

[1278] Recording user interactions

[1279] When a user interacts with the advertisements or information provided, their actions are recorded. The input data is a log of the user's interaction (for example, tapping on a push notification to view more information).

[1280] The device sends this interaction data to the server. The output data is an interaction log in JSON format. Specifically, the device detects interaction events and sends the data to the server in real time.

[1281] Step 10:

[1282] Data Feedback

[1283] The server analyzes the user's interaction data and uses it to generate personalized ads and local information for the next visit. The input data is the collected interaction data.

[1284] The server analyzes this data to identify emerging trends and patterns. The output is an optimization profile for the next ad or information offering. Specifically, machine learning algorithms are used to analyze the interaction data and use the results to improve the accuracy of future ads and information offerings. Emotional feedback is also included in the analysis.

[1285] (Application example 2)

[1286] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1287] Conventional advertising systems have a low level of personalization accuracy, making it difficult to provide users with timely and useful information. Furthermore, delivering ads uniformly without considering the user's current emotional state can be counterproductive. For example, if an ad is sent to a user when they are tired, it may not only not pique their interest, but may also create a negative impression of the ad itself. Furthermore, there is no established method for effectively utilizing real-time location information and emotional data, making it difficult to provide information tailored to local trends and users' interests and preferences.

[1288] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for transmitting user data and emotion data to the server, means for analyzing regional trends based on the user data and emotion data, and means for inputting the analysis results into a generative AI model and generating personalized advertisements or regional information based on the user's current location and emotional state. This makes it possible to provide timely and highly accurate personalized information that takes into account the user's current location and emotional state.

[1289] "User Data" is a general term for all data related to a user, such as the user's location information, interests and preferences, and behavioral history.

[1290] "Emotional data" is information about a user's emotional state at any given time, obtained by analyzing their facial expressions, tone of voice, and other biometric signals.

[1291] A "server" is a computer system that processes various data, such as analyzing user data and emotional data, generating advertisements, and providing information.

[1292] "Local trends" refers to information that shows trends and patterns regarding user interests, popular events, facilities, etc. within a particular geographic area.

[1293] "Generative AI models" refer to artificial intelligence algorithms and learning models used to generate personalized ads and location information using user and emotional data as input.

[1294] "Personalized advertising or location information" refers to advertising or location information that is individually optimized based on a user's individual interests, preferences, behavioral history, and emotional state.

[1295] "User terminal" refers to a communication device used by a user, such as a smartphone or tablet.

[1296] "Display device" refers to a device, such as an electronic bulletin board or digital signage, that visually presents advertisements or information to users.

[1297] "Push notification" is a messaging service that sends information from a server to a user's device in real time.

[1298] "Interact" means that a user interacts in some way with the advertisement or location information provided.

[1299] This invention is a system that collects and analyzes user location information, interests, preferences, behavioral data, and emotional data, and provides personalized advertisements and local information in real time. This system operates mainly using the following hardware and software.

[1300] Hardware

[1301] 1. User device: a smartphone or tablet, including a GPS and a camera.

[1302] 2. Display devices: digital signage and electronic bulletin boards.

[1303] software

[1304] 1. Emotion Engine: Facial expression recognition and voice analysis software to capture the user's emotional state.

[1305] 2. Generative AI model: Runs on the server and generates personalized ads and local information based on user data and emotional data.

[1306] 3. Data transmission / reception module: A communication module for exchanging data between the user terminal and the server.

[1307] Process Overview

[1308] 1. Acquisition and transmission of user data

[1309] When a user launches a smartphone app, the device's GPS function is used to obtain current location information. The app also collects the user's interests, preferences, and behavioral history. This data is sent to a server in real time.

[1310] 2. Acquiring Emotion Data

[1311] The smartphone is equipped with an emotion engine that analyzes the user's facial expressions and tone of voice, and this acquires emotional data, which is also sent to the server.

[1312] 3. Data analysis and ad generation

[1313] On the server side, user and sentiment data is analyzed to understand specific local trends and user interests, which is then used by generative AI models to generate personalized ads and local information.

[1314] 4. Provision of Information

[1315] The generated advertisements and local information are sent in real time to the user's device or display device, and on smartphones, the information is provided via push notifications.

[1316] Specific examples

[1317] For example, if a user is in a shopping mall, the generative AI model can generate special sale information and coupons for the mall based on the smartphone's location information, collected interest and preference data, and the relaxed state detected by the emotion engine, and provide them to the user via push notification.

[1318] Prompt Sentence Examples

[1319] "Use up-to-date sentiment data to generate ads that are best suited to relaxed users in a specific location."

[1320] This will enable us to provide users with the information they need in a timely manner, improving the effectiveness of our advertising. In addition, by collecting user feedback data, we will be able to further improve the accuracy of the information we provide next time.

[1321] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1322] Step 1:

[1323] When a user launches a smartphone app, the device uses its GPS function to obtain the user's current location. This location information is input data that the application sends to the server as user data. This location information is used to inform the server of the user's current location. Specifically, the smartphone's GPS sensor obtains the location information, and the application sends it to the server via an HTTP request.

[1324] Step 2:

[1325] The user's interests, preferences, and past behavioral data are also collected at the same time. This information, such as the user's past search keywords and visited places, is sent to the server along with the interest and preference data entered within the app. This input data forms an individual user profile and is used by the server to analyze the user's interests. Specifically, past history data is collected from the application's database and sent to the server along with location information.

[1326] Step 3:

[1327] The device is equipped with an emotion engine that analyzes the user's facial expressions (using the camera) and voice tone (using the microphone) to obtain emotion data. This emotion data is also sent to the server. The emotion data as input data indicates the user's current emotional state and is used to personalize advertisements. Specifically, the device uses the smartphone's camera and microphone to collect the user's biometric data, and the results analyzed by the emotion engine are sent to the server.

[1328] Step 4:

[1329] The server analyzes the received location information, interest and preference data, behavioral data, and emotional data. For this analysis, it uses big data analytics technology to gain insights based on local trends and user interests. This process includes data processing and calculation, such as data retrieval from the database, cleansing, point analysis, and trend analysis. Specifically, it retrieves trend data from the database and extracts information on popular events and stores by region.

[1330] Step 5:

[1331] The server inputs the analysis results into a generative AI model to generate personalized ads or local information based on the user's current location and emotional state. In this generation process, the generative AI model generates the optimal ads or information based on the input data and generates the results. Specifically, the generative AI model matches the user's profile with trend data to generate the optimal ads or information. An example of a prompt is, "Use the latest emotional data to generate the optimal ads for relaxed users in a specific area."

[1332] Step 6:

[1333] The generated advertisements and local information are sent from the server to the user's device or display device. In the case of smartphones, the information is provided to the user via push notification. The generated results as input data are sent to the device as an HTTP response, and the device displays the push notification. Specifically, the server returns the generated results as an HTTP response, and the smartphone receives the data and triggers the push notification.

[1334] Step 7:

[1335] When a user interacts with the generated advertisement or local information, the behavioral data is recorded and sent to the server. This behavioral data includes the click history of push notifications and the time spent viewing advertisements. Specifically, the system captures user operation events and sends them to the server as behavioral data.

[1336] Step 8:

[1337] The server analyzes the interaction data and uses it to generate the next personalized advertisements and local information. This analysis process includes analyzing past behavioral patterns and evaluating emotional data. Specifically, the behavioral history stored in the database is analyzed and used as input data for the next generation AI model.

[1338] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1339] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1340] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1341] [Fourth embodiment]

[1342] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1343] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1344] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1345] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1346] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1347] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1348] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1349] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1350] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1351] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1352] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1353] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1354] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1355] MODE FOR CARRYING OUT THE INVENTION

[1356] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. This system works through a server, terminals, and records of user interactions.

[1357] Retrieving User Information

[1358] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[1359] Big Data Analysis

[1360] The server analyzes the collected user data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[1361] Ad generation by generative AI

[1362] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history.

[1363] Real-time information distribution

[1364] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by.

[1365] Recording user interactions

[1366] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided.

[1367] Specific examples

[1368] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The generation AI uses this information to create an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, coupons for nearby restaurants are also generated and provided at the same time based on the user's past usage history. The user receives this information, and the data on their interactions is sent back to the server to be used to provide information in the future.

[1369] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[1370] The processing flow will be explained below.

[1371] Specific processing steps of the program

[1372] Information Acquisition Phase

[1373] Step 1:

[1374] The user launches the smartphone app and logs in.

[1375] The device uses GPS to obtain the user's current location.

[1376] Step 2:

[1377] The location information acquired by the device is sent to the server.

[1378] The server receives the location information and determines the user's location.

[1379] Step 3:

[1380] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[1381] The device sends this selection data to the server.

[1382] Step 4:

[1383] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[1384] The server receives the behavioral data and stores it in a database.

[1385] Data analysis phase

[1386] Step 5:

[1387] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[1388] The server also collects social media data and local pedestrian flow data.

[1389] Step 6:

[1390] The server cleanses the data and formats it into an easy-to-parse format.

[1391] Incomplete or incorrect data is filtered out.

[1392] Step 7:

[1393] The server analyzes regional trends.

[1394] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[1395] Ad generation phase

[1396] Step 8:

[1397] The server inputs the analysis results into the generative model.

[1398] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, and trend data.

[1399] Step 9:

[1400] The server utilizes the generative model to generate personalized advertisements or local information.

[1401] For example, if a user is near a museum, ads for art exhibitions will be generated.

[1402] Step 10:

[1403] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[1404] The server finalizes the advertisement and information.

[1405] Information distribution phase

[1406] Step 11:

[1407] The server sends the generated advertisements and local information to the user's terminal.

[1408] The device will display information to the user via push notifications or at specific times.

[1409] Step 12:

[1410] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[1411] Devices and electronic billboards display information relevant to the user's current location.

[1412] Interaction Recording Phase

[1413] Step 13:

[1414] The device records how the user interacts with advertisements and information.

[1415] For example, if a user taps on a push notification to view more information, that action is recorded.

[1416] Step 14:

[1417] The terminal transmits the recorded interaction data to the server.

[1418] The server receives the interaction data and stores it in a database.

[1419] Step 15:

[1420] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[1421] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[1422] Through these steps, users will be provided with personalized advertisements and local information in real time, creating a system that will contribute to revitalizing the local economy and promoting tourism.

[1423] Example 1

[1424] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1425] Conventional ad delivery systems are unable to fully utilize users' real-time interests, preferences, location information, and behavioral data, making it difficult to provide truly useful personalized ads and local information to users. Furthermore, there is a lack of technology to effectively analyze this data and provide information at the optimal time. Furthermore, there are few ways to improve the accuracy of subsequent ads and information based on user responses.

[1426] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1427] In this invention, the server includes: means for acquiring user location information; means for transmitting user data, including the acquired location information and interest and preference data, to the server; means for transmitting user behavior data from the terminal to the server; means for the server to analyze regional trends based on the user data and external data using a big data analysis tool; means for inputting the analysis results into a generative AI model to generate personalized advertisements or regional information; means for transmitting the generated advertisements or regional information to the user terminal or display device; means for recording the user's behavior when interacting with the generated advertisements or regional information and transmitting the behavior data to the server; and means for the server to analyze the interaction data and use it to generate the next personalized advertisement or regional information. This makes it possible to effectively utilize the user's real-time location information, interest and preference data, and behavior data to provide personalized information. Furthermore, the accuracy of the service can be further improved based on user feedback.

[1428] "User" refers to an individual or entity that uses the System, specifically the entity that provides location information, interest and preference data, and behavioral data.

[1429] "Location Information" means data that indicates a user's current geographic location, obtained using technologies such as GPS.

[1430] "Interest and Preference Data" refers to information about your interests and preferences that you enter within the App.

[1431] "Behavioral data" refers to data including the history of actions taken by a user within an app, such as search history and browsing history.

[1432] "Server" refers to a computer system that receives, stores, and analyzes data sent by users, and generates and distributes advertisements and local information.

[1433] "Big data analytics tools" are software and technologies that allow for the efficient analysis of vast amounts of data, specifically those used to identify regional trends and demand.

[1434] "Generative AI model" refers to artificial intelligence technology that uses pre-trained algorithms to generate personalized ads and local information based on prompts.

[1435] A "prompt" is an instruction entered into a generative AI model, containing a specific request to generate a particular advertisement or local information.

[1436] "User terminal" refers to a device such as a smartphone or tablet used by a user, which obtains location information and receives advertisements.

[1437] "Display device" refers to a device such as an electronic signboard or display for displaying information from the system.

[1438] "Interaction Data" is data that indicates how a user interacts with an advertisement or local information and is sent to a server.

[1439] MODE FOR CARRYING OUT THE INVENTION

[1440] This invention is a system that provides personalized advertisements and local information in real time based on users' location information, interests, preferences, and behavioral data. The system functions through a server, terminals, and records of user interactions.

[1441] Retrieving User Information

[1442] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The user can also input their interests and preferences within the app, and this interest and preference data is also sent to the server. Furthermore, user behavior data, such as search history and browsing history within the app, is also collected and sent to the server.

[1443] Big Data Analysis

[1444] The server uses big data analysis tools (e.g., Apache Hadoop) to identify local trends and demand based on the received user data. Analysis includes social media mention counts and foot traffic data. The results of this analysis are fed into a generative AI model (e.g., OpenAI GPT-4).

[1445] Ad generation by generative AI

[1446] The server creates a prompt based on the analysis results and inputs it into the generative AI model. For example, a prompt such as "Please generate an advertisement for a popular art event in Kyoto" is input into the generative AI model. The generative AI model generates personalized advertisements and local information based on this prompt.

[1447] Real-time information distribution

[1448] The generated advertisements and local information are sent from the server to the user's smartphone via push notifications. Using communication technologies such as Firebase Cloud Messaging, the information is delivered to the user instantly. The information is also displayed on display devices (e.g., electronic billboards) when the user passes nearby.

[1449] Recording user interactions

[1450] When a user interacts with an advertisement or information, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is generated.

[1451] Specific examples

[1452] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, the app uses GPS to obtain their current location (Kyoto) and sends it to the server. The server then analyzes social media data and people flow data to determine that the "Kyoto Art Festival" is popular. The server then inputs a prompt message, "Please generate an advertisement for the Kyoto Art Festival," into the generative AI model, which then generates an advertisement for the festival. The generated advertisement is sent from the server to the user's smartphone via push notification, allowing the user to receive the information immediately. In addition, similar information is displayed in real time on nearby electronic billboards. When the user taps the push notification to view more information, their behavioral data is sent to the server and used to generate future advertisements.

[1453] In this way, the system can provide users with useful information in real time, contributing to the revitalization of the local economy and tourism. It can also continuously collect user feedback to improve the accuracy of the information it provides.

[1454] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1455] Step 1:

[1456] Retrieving User Information

[1457] When a user launches a smartphone app, the device uses GPS to obtain current location information. This location information is then sent from the device to a server. Specifically, the device receives input data from the GPS and sends it to the server in real time. The output is location data.

[1458] Step 2:

[1459] Enter your interests and preferences

[1460] The user enters their interests and preferences within the app. This data is also sent to the server. Specifically, the device receives the user's input as text data and sends it to the server. The output is interest and preference data.

[1461] Step 3:

[1462] Behavioral data collection

[1463] Every time a user searches or browses within the app, the device records the behavioral data and sends it to the server. Specifically, the device captures the user's action events (search history, browsing history, etc.) and sends them to the server. The output is behavioral data.

[1464] Step 4:

[1465] Data collection

[1466] The server receives location information, interest and preference data, and behavioral data. Specifically, the server acquires this data from each device and stores it in a database. The input is the data sent from the device, and the output is organized user data.

[1467] Step 5:

[1468] Big Data Analysis

[1469] The server uses big data analysis tools (e.g., Apache Hadoop) to identify regional trends and demand. The server analyzes data such as the number of mentions on social media and foot traffic data to extract trend information for each region. The input is user data and external data, and the output is the analysis results.

[1470] Step 6:

[1471] Generate prompt statement

[1472] The server creates a prompt based on the analysis results. Specifically, it generates a text-based instruction to be input into the generative AI model. For example, a prompt might be generated saying, "Please generate an advertisement for a popular art event in Kyoto." The input is the analysis results, and the output is the prompt.

[1473] Step 7:

[1474] AI-powered ad generation

[1475] The prompt text generated by the server is input into a generative AI model (e.g., OpenAI GPT-4) to generate personalized advertisements and local information. Specifically, the AI ​​analyzes the prompt text and generates advertisements and information based on it. The input is the prompt text, and the output is the generated advertisements and local information.

[1476] Step 8:

[1477] Real-time information distribution

[1478] The server sends the generated ads and location information to the user's smartphone via push notification. The information is delivered in real time using Firebase Cloud Messaging. The input is the generated ads and location information, and the output is the push notification.

[1479] Step 9:

[1480] Electronic bulletin board update

[1481] The server sends information to a nearby display device, which displays relevant advertisements and information as the user passes by. The input is the generated advertisement or information, and the output is the display on the display device.

[1482] Step 10:

[1483] Recording user interactions

[1484] Every time a user taps on a push notification to view more information, the device records that action and sends it to the server. The input is the user interaction, and the output is the recorded behavior data.

[1485] Step 11:

[1486] Utilizing data to generate the next ad

[1487] The server analyzes the interaction data and uses it to generate the next personalized advertisement and local information. Specifically, it generates more accurate prompts and advertisements based on past user behavior data. The input is the interaction data, and the output is the analysis results that will be used to generate the next advertisement.

[1488] (Application example 1)

[1489] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1490] While modern autonomous vehicles have advanced in technology to improve safety and efficiency, they still face limitations in providing personalized information and entertainment to passengers. In particular, in tourism and commercial use, there is a lack of ways for passengers to receive information tailored to their interests and preferences in real time to enhance their experience. There is also a need for a mechanism to utilize user feedback on the information provided to provide more highly personalized services.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1492] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information to the server, means for the server to analyze regional trends based on the user data, means for inputting the analysis results into a generative AI model to generate personalized tourist spot information or advertisements, means for transmitting the generated tourist spot information or advertisements to a user terminal or display device in the vehicle, means for recording user behavior when interacting with the generated tourist spot information or advertisements and transmitting the behavioral data to the server, and means for the server to analyze the interaction data and use it to generate the next personalized tourist spot information or advertisements. This allows passengers to receive personalized information in real time while in an autonomous vehicle, improving their experience and continuously improving the accuracy of the information provided.

[1493] An "autonomous vehicle" is a vehicle that does not require a driver and operates using an automatic control system.

[1494] "Location Information" means data that indicates a specific location using GPS or other location measurement technology.

[1495] "User Data" refers to data that includes general information about a user, such as the user's location, interests, preferences, and behavioral data.

[1496] "Server" means a computer system that collects, analyzes, processes data, and distributes the generated information.

[1497] "Regional trends" refers to information about events, places, and other things that are currently popular or attracting attention in a particular region.

[1498] "Analysis" is the process of extracting useful information and patterns from collected data.

[1499] A "generative AI model" is an algorithm that uses artificial intelligence to generate personalized information from input data.

[1500] "Tourist spot information" refers to detailed information about tourist spots.

[1501] An "advertisement" is an informational message created to promote a particular product, service, etc.

[1502] A "user device" is a device that can be directly operated by a user, such as a smartphone or tablet.

[1503] A "display device" is a display or screen used to visually present information to a user.

[1504] "Interact" means that a user interacts in some way with the information or application provided.

[1505] "Behavioral Data" means data that describes how a user interacts with an application or the information provided to them.

[1506] This invention aims to realize a system for providing personalized information to passengers in an autonomous vehicle. Specifically, the system is constructed by combining the following steps.

[1507] First, the server uses GPS to obtain the location information of the autonomous vehicle. The GPS module is installed in the vehicle and periodically determines its current location. This location information is sent to the server.

[1508] The server then obtains passengers' interests and preferences through user devices such as tablets and smartphones. Passengers input their interests and preferences through tablets installed in the vehicle or through an app on their own smartphone. This data is also sent to the server and stored as user data.

[1509] The server also collects behavioral data, such as user search and browsing history within the app. Based on this data, the server analyzes trends and demand by region. A big data analysis system (e.g., Apache Spark) is used for the analysis, and the popularity and key points of events in a specific region are identified based on the number of mentions on social media.

[1510] The server then inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate personalized tourist spot information and advertisements. An example of a specific prompt might be, "The user is interested in historical buildings. Please tell me about popular historical buildings as the latest tourist attractions near their current location."

[1511] The generated tourist information and advertisements are sent in real time to tablets and smartphones on board, giving passengers instant access to personalized information.

[1512] Finally, when a passenger interacts with the displayed information (e.g., tapping on a coupon), their actions are again sent to the server, which analyzes this interaction data and uses it the next time the information is provided, further improving the accuracy and personalization of the information provided.

[1513] For example, if a passenger expresses an interest in "historical buildings," the server analyzes local trends based on the acquired location information and generates information and coupons related to "historical castles," a popular tourist spot nearby. This information is displayed in real time on the tablet, and when the passenger uses the coupon, the data is sent back to the server to help personalize the service for future visits.

[1514] In this way, the invention can improve the passenger experience in autonomous vehicles and continuously improve the accuracy of the information provided.

[1515] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1516] Step 1:

[1517] (Processing content) Acquisition and transmission of location information

[1518] (Specific operation) The GPS module installed in the autonomous vehicle periodically acquires its current location. This location information is sent to a server via a communication module in the vehicle.

[1519] (Input) Current position from the GPS module

[1520] (Data processing) Format conversion of location information

[1521] (Output) Location data sent to the server

[1522] Step 2:

[1523] (Processing content) Acquiring user interests and preferences

[1524] (Specific operation) The user inputs their interests and preferences using a tablet installed in the vehicle or an app on the passenger's smartphone. The input data is sent from the app to the server.

[1525] (Input) Interest and preference data from tablet and smartphone apps

[1526] (Data processing) Linking user ID with interest and preference data

[1527] (Output) User interest and preference data sent to the server

[1528] Step 3:

[1529] (Processing content) Collection of user behavior data

[1530] (Specific operation) Collects behavioral data such as search history and browsing history within the app and sends it to the server. To do this, the app generates log data and periodically sends it to the server.

[1531] (Input) Search history and browsing history within the app

[1532] (Data processing) Standardizing the format of behavioral data

[1533] (Output) Action log data sent to the server

[1534] Step 4:

[1535] (Processing content)Analysis of regional trends

[1536] (Specific operation) The server uses the collected user data and location information to analyze regional trends using a big data analysis system (e.g., Apache Spark). Trends are identified based on data such as the number of mentions on social media and the flow of people.

[1537] (Input) User data, location information

[1538] (Data processing) Big data aggregation and statistical analysis

[1539] (Output) Regional trend data as analysis results

[1540] Step 5:

[1541] (Processing content) Creation of personalized information

[1542] (Specific operation) The server inputs the analysis results into a generative AI model (e.g., OpenAI's GPT-4) to generate tourist spot information and advertisements appropriate for the user. A prompt sentence is input into the generative AI model, and it is generated based on this.

[1543] (Input) Regional trend data, prompt statement

[1544] (Data processing) Information generation using generative AI models

[1545] (Output) Personalized information (tourist spot information, advertisements)

[1546] Step 6:

[1547] (Processing content) Real-time information distribution

[1548] (Specific operation) The generated personalized information is sent to tablets and smartphones in the vehicle and notifications are sent, allowing passengers to receive information in real time.

[1549] (Input) Personalization Information

[1550] (Data processing) Creation and sending of notification messages

[1551] (Output) Information displayed on the user's terminal

[1552] Step 7:

[1553] (Processing content) Recording of user interactions

[1554] (Specific actions) When the user interacts with the displayed information (tap, click, use, etc.), that action is sent back to the server, which stores this data and uses it for the next analysis.

[1555] (Input) User interaction data

[1556] (Data processing) Recording interaction data and integrating it into analytical data

[1557] (Output) Interaction data stored on the server

[1558] Step 8:

[1559] (Processing content) Analysis for the next information provision

[1560] (Specific operation) The server analyzes the interaction data and uses it to generate personalized information for the next visit. This analysis makes it possible to provide more accurate information.

[1561] (Input) Interaction data

[1562] (Data processing) Modify the model for the next generation of personalized information

[1563] (Output) Analysis data for the next information provision

[1564] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1565] MODE FOR CARRYING OUT THE INVENTION

[1566] This invention is a system that provides personalized advertisements and local information in real time using an emotion engine that recognizes user emotions in addition to user location information, interests, preferences, and behavioral data. This system functions through a server, terminals, the emotion engine, and records of user interactions.

[1567] Retrieving User Information

[1568] When a user uses a smartphone app, the device uses GPS to obtain current location information. This location information is sent to a server. The app also allows users to input their interests and preferences. This data is also sent to the server and saved as a user profile. In addition, user behavior data, such as search history and browsing history within the app, is also collected.

[1569] Acquiring emotion data

[1570] The device is equipped with an emotion engine that analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data, which is then sent to a server in real time and stored as the user's emotional profile.

[1571] Big Data Analysis

[1572] The server analyzes the collected user data and sentiment data to identify trends and demand in each region. By analyzing data including the number of mentions on social media and the flow of people, it is possible to understand the popularity of events and new points of interest in a specific region. The results of this analysis are input into a generative model.

[1573] Ad generation by generative AI

[1574] The server inputs the analysis results into a generative model and uses AI technology to generate personalized advertisements and local information. For example, if a user is interested in art museums, the system can generate advertisements for art exhibitions in the area. It can also generate coupons for specific restaurants based on the user's past behavioral history and current emotional state.

[1575] Real-time information distribution

[1576] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access it immediately. On electronic billboards, relevant information is displayed in real time as the user passes by. In addition, the timing of information delivery can be adjusted based on the user's emotions recognized by the emotion engine. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[1577] Recording user interactions

[1578] When a user interacts with an advertisement or information provided through an app or signage, that behavior is recorded. For example, if a user taps on a push notification to view more information, that behavior is sent to the server. This allows for more accurate personalization the next time an advertisement or information is provided. Emotional data evoked by the user's interaction is also recorded and sent to the server as feedback.

[1579] Specific examples

[1580] As a concrete example, consider the case where a user visits Kyoto. When the user launches a smartphone app, their current location information is sent to the server. The server analyzes trends in Kyoto based on social media data and people flow data, and determines that the "Kyoto Art Festival" is popular. The emotion engine then recognizes the user's relaxed emotions. Based on this information, the generative AI creates an advertisement for the festival and sends it to the user's smartphone via push notification. Furthermore, based on the user's past usage history and current emotional state, coupons for nearby restaurants are also generated and provided at the same time. The user receives this information, and the data on their interactions is sent back to the server to be used for providing information in the future.

[1581] In this way, the system can provide users with useful information in real time, contributing to regional economic revitalization and tourism promotion. It can also continuously collect user feedback to improve the accuracy of the information provided. The introduction of an emotion engine enables more advanced personalization based on the user's emotional state, further improving the effectiveness of information provision.

[1582] The processing flow will be explained below.

[1583] Specific processing steps of the program

[1584] Information Acquisition Phase

[1585] Step 1:

[1586] The user launches the smartphone app and logs in.

[1587] The device uses GPS to obtain the user's current location.

[1588] Step 2:

[1589] The location information acquired by the device is sent to the server.

[1590] The server receives the location information and determines the user's location.

[1591] Step 3:

[1592] Users select categories of interest within the app (e.g., food, tourist attractions, events).

[1593] The device sends this selection data to the server.

[1594] Step 4:

[1595] The device collects user behavioral data (such as search history and browsing history within the app) and sends it to the server.

[1596] The server receives the behavioral data and stores it in a database.

[1597] Emotion data acquisition phase

[1598] Step 5:

[1599] The device uses a camera and microphone to detect the user's facial expressions and tone of voice.

[1600] The emotion engine analyzes this data to recognize the user's emotions.

[1601] Step 6:

[1602] The recognized emotion data is sent to the server in real time.

[1603] The server receives the emotion data and adds it to the user profile.

[1604] Data analysis phase

[1605] Step 7:

[1606] The server analyzes the location, interest and preference data, and behavioral data collected from users.

[1607] The server also collects social media data and local pedestrian flow data.

[1608] Step 8:

[1609] The server cleanses the data and formats it into an easy-to-parse format.

[1610] Incomplete or incorrect data is filtered out.

[1611] Step 9:

[1612] The server analyzes regional trends.

[1613] The popularity of a particular event or tourist destination is determined based on how often it is mentioned on social media and the number of visitors.

[1614] Ad generation phase

[1615] Step 10:

[1616] The server inputs the analysis results into the generative model.

[1617] The data provided to the generative model includes the user's current location, interests and preferences, behavioral history, emotional data, and trend data.

[1618] Step 11:

[1619] The server utilizes the generative model to generate personalized advertisements or local information.

[1620] For example, if a user is near a museum, ads for art exhibitions will be generated.

[1621] Step 12:

[1622] The server verifies the content of the advertisements and information generated by the generative model and corrects it if necessary.

[1623] The server finalizes the advertisement and information.

[1624] Information distribution phase

[1625] Step 13:

[1626] The server sends the generated advertisements and local information to the user's terminal.

[1627] The device will display information to the user via push notifications or at specific times.

[1628] Step 14:

[1629] The server displays advertisements and information in real time on an electronic billboard as the user passes by at the appropriate time.

[1630] Devices and electronic billboards display information relevant to the user's current location.

[1631] Step 15:

[1632] Based on the emotional data, the server adjusts the timing of information distribution.

[1633] For example, it could deliver restaurant promotion information when the user is relaxing.

[1634] Interaction Recording Phase

[1635] Step 16:

[1636] The device records how the user interacts with advertisements and information.

[1637] For example, if a user taps on a push notification to view more information, that action is recorded.

[1638] Step 17:

[1639] The terminal transmits the recorded interaction data to the server.

[1640] The server receives the interaction data and stores it in a database.

[1641] Step 18:

[1642] The server analyzes the interaction data and uses it for the next advertisement or information provision.

[1643] The server then feeds the analysis results back into the generative model to further optimize the delivery of information to users.

[1644] Through these steps, a system will be built that provides users with personalized advertisements and local information in real time, contributing to the revitalization of local economies and the promotion of tourism. Furthermore, the introduction of an emotion engine will provide more appropriate information based on the user's emotional state, further improving the effectiveness of information provision.

[1645] Example 2

[1646] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1647] Conventional personalized advertising and local information provision systems typically generate advertisements and information based on users' location information, interests, preferences, and behavioral data. However, personalization that takes into account the user's emotional state is not adequate, and the timing and content are often inappropriate. As a result, there are issues with not being able to attract users' attention and reducing the effectiveness of advertisements and information.

[1648] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1649] In this invention, the server includes means for acquiring user location information, means for transmitting user data including the acquired location information, interests, preferences, and behavioral data to the server, means for acquiring user emotion data using an emotion engine installed in the terminal, means for transmitting the acquired emotion data to the server, means for the server to analyze regional trends based on the user data and emotion data and generate a prompt message using a generative AI model, means for generating personalized advertisements or regional information based on the generated prompt message, means for transmitting the generated advertisements or regional information to the user terminal or electronic billboard, and means for adjusting the timing of information delivery based on the user's emotional state, thereby enabling the provision of more effective personalized advertisements and regional information according to the user's emotional state.

[1650] "Location information" is geographic coordinate data that indicates where a user is currently located.

[1651] "Interests and Preferences" refers to information about a user's preferred activities, places, items, etc.

[1652] "Behavioral data" refers to information such as search history, browsing history, and click history when a user uses the system.

[1653] "Emotional data" refers to information about a user's emotional state, analyzed from facial expressions, tone of voice, etc.

[1654] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[1655] A "server" is a remote computer system that receives and analyzes user data and emotional data.

[1656] A "generative AI model" is a machine learning model that generates prompts based on user data and emotional data to create personalized advertisements and information.

[1657] A "prompt sentence" is an input sentence that a generative AI model uses to generate personalized ads or information.

[1658] An "advertisement" is text, image, or video that provides commercial information to users.

[1659] "Local information" is information about events and places associated with a particular geographic area.

[1660] A "push notification" is a notification message that provides information to a user's device in real time.

[1661] An "electronic billboard" is an electronic display device for displaying advertisements and information in a specific area.

[1662] "Interact" refers to the user's reaction to and manipulation of the advertisements and information provided.

[1663] "Interaction data" is a record of the actions and behaviors that users take in response to advertisements or information.

[1664] "Feedback" is information used to improve the system's personalization accuracy based on user emotional and interaction data.

[1665] This invention is a system that uses user location information, interests, preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time. This system is mainly implemented using a server, a terminal, an emotion engine, and a generative AI model. Detailed embodiments of the system are described below.

[1666] Retrieving User Information

[1667] When a user launches the smartphone app, the device uses GPS to obtain its current location information, which is then captured as latitude and longitude data and sent to the server in JSON format.

[1668] Next, users enter their interests and preferences within the app. For example, they enter information such as their interest in museums and restaurants in text format, and this data is also sent to the server. In addition, the device collects behavioral data such as search history, browsing history, and click history within the app, and this information is also sent to the server in JSON format.

[1669] Acquiring emotion data

[1670] The device is equipped with an emotion engine that captures the user's emotional data. The emotion engine uses a camera to capture the user's facial expressions and analyzes their emotional state using a facial recognition algorithm. It also uses a microphone to capture the tone of voice and analyzes it using a voice recognition algorithm. The captured emotion data is sent to a server in real time and stored as the user's emotional profile.

[1671] Big Data Analysis

[1672] The server integrates the acquired user data and sentiment data to analyze trends and demand in each region. Specifically, it analyzes big data including the number of mentions on social media and population flow data. This analysis can, for example, identify the popularity of events in a specific region and identify new points of interest.

[1673] Ad generation by generative AI

[1674] The server inputs the analysis results into a generative AI model to generate personalized advertisements and local information. For example, if a user is interested in museums, the generative AI model will generate advertisements based on the following prompt:

[1675] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[1676] The generative AI model uses this prompt to create the most relevant ads and local information for the user.

[1677] Real-time information distribution

[1678] The generated advertisements and local information are sent from the server to the user's smartphone or electronic billboard. On smartphones, information is provided via push notifications, allowing users to access the information instantly. On electronic billboards, relevant information is displayed in real time as the user passes by. The emotion engine also adjusts the timing of information delivery based on the user's emotional state. For example, restaurant promotions can be displayed when the user is in a relaxed state.

[1679] Recording user interactions

[1680] When a user interacts with an advertisement or information provided through an app or electronic billboard, that behavior is recorded and sent to a server. For example, if a user taps on a push notification to view more information, that behavioral data is sent to the server. This interaction data is used to provide advertisements and information in the future, enabling more accurate personalization.

[1681] In this way, the system integrates users' location information, interests and preferences, behavioral data, and emotional data to provide personalized advertisements and local information in real time, enabling it to accurately deliver useful information to users.

[1682] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1683] Step 1:

[1684] Retrieving User Information

[1685] The device uses GPS to obtain the user's current location information. The input data is the latitude and longitude received from the GPS sensor. The output data is the obtained location information (latitude and longitude) converted to JSON format.

[1686] The device sends this location information to the server. Specifically, the device sends the location information to the server using an HTTP POST request.

[1687] Step 2:

[1688] Enter your interests and preferences

[1689] Users input their interests and preferences into a smartphone app. The input data is information about interests and preferences that users enter by tapping or entering text.

[1690] The device converts this information into JSON format and sends it to the server. The output data is JSON format data of the user's interests and preferences. Specifically, the device collects data entered through the user interface and sends it to the server.

[1691] Step 3:

[1692] Behavioral data collection

[1693] The device collects the user's search history, browsing history, and click history within the app. The input data is a log of the user's actions.

[1694] The device converts this data into JSON format and sends it to the server. The output data is JSON formatted behavioral data. Specifically, it monitors the action log and sends the data to the server in real time. As with other data, it uses an HTTP POST request.

[1695] Step 4:

[1696] Acquiring emotion data

[1697] The emotion engine installed on the device analyzes the user's facial expressions. The input data is image data of the user's face captured by the camera. The output data is data indicating the analyzed emotional state (e.g., smiling, angry, surprised, etc.).

[1698] The device sends this data to a server, which uses a facial recognition algorithm to analyze the user's emotional state and sends the results to the server via an HTTP POST request.

[1699] Step 5:

[1700] Voice Tone Analysis

[1701] The device analyzes the tone of the user's voice. The input data is the voice data captured by the microphone. The output data is data indicating the analyzed emotional state (e.g., stressed state, relaxed state, etc.).

[1702] The device sends this data to the server, which uses a speech recognition algorithm to analyze the tone of the voice and sends the results to the server via an HTTP POST request.

[1703] Step 6:

[1704] Big Data Analysis

[1705] The server integrates the received user data and sentiment data to analyze local trends and demand. The input data is the collected user data and sentiment data. Additional big data (e.g., social media mention counts and population mobility data) is also used for analysis.

[1706] The server analyzes this data to identify trends and demand in specific regions. The output data is data showing trends and demand by region. Specifically, it runs big data analysis algorithms to identify trends by region.

[1707] Step 7:

[1708] Ad generation by generative AI

[1709] The server generates prompt sentences for the generative AI model based on the results of big data analysis. The input data is the analysis results, the user's interests and preferences, behavioral data, and emotional data. Examples of prompt sentences are:

[1710] "The user is currently in Kyoto and is interested in museums. Based on their past search history, we can infer that they are relaxing. Suggest promotions for nearby events and restaurants."

[1711] The generative AI model generates personalized advertisements and local information based on this prompt. The output data is personalized advertisements and local information. Specifically, the generative AI model analyzes the prompt and generates appropriate advertisements and information.

[1712] Step 8:

[1713] Real-time information distribution

[1714] The server transmits the generated advertisement and area information to the user terminal or the electronic billboard, and the input data is the generated advertisement and area information.

[1715] The server delivers this data in real time, providing information to users via push notifications on smartphones and real-time displays on electronic billboards. Specifically, the server sends the generated advertisements and local information to the delivery system, triggering push notifications.

[1716] Step 9:

[1717] Recording user interactions

[1718] When a user interacts with the advertisements or information provided, their actions are recorded. The input data is a log of the user's interaction (for example, tapping on a push notification to view more information).

[1719] The device sends this interaction data to the server. The output data is an interaction log in JSON format. Specifically, the device detects interaction events and sends the data to the server in real time.

[1720] Step 10:

[1721] Data Feedback

[1722] The server analyzes the user's interaction data and uses it to generate personalized ads and local information for the next visit. The input data is the collected interaction data.

[1723] The server analyzes this data to identify emerging trends and patterns. The output is an optimization profile for the next ad or information offering. Specifically, machine learning algorithms are used to analyze the interaction data and use the results to improve the accuracy of future ads and information offerings. Emotional feedback is also included in the analysis.

[1724] (Application example 2)

[1725] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1726] Conventional advertising systems have a low level of personalization accuracy, making it difficult to provide users with timely and useful information. Furthermore, delivering ads uniformly without considering the user's current emotional state can be counterproductive. For example, if an ad is sent to a user when they are tired, it may not only not pique their interest, but may also create a negative impression of the ad itself. Furthermore, there is no established method for effectively utilizing real-time location information and emotional data, making it difficult to provide information tailored to local trends and users' interests and preferences.

[1727] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for transmitting user data and emotion data to the server, means for analyzing regional trends based on the user data and emotion data, and means for inputting the analysis results into a generative AI model and generating personalized advertisements or regional information based on the user's current location and emotional state. This makes it possible to provide timely and highly accurate personalized information that takes into account the user's current location and emotional state.

[1728] "User Data" is a general term for all data related to a user, such as the user's location information, interests and preferences, and behavioral history.

[1729] "Emotional data" is information about a user's emotional state at any given time, obtained by analyzing their facial expressions, tone of voice, and other biometric signals.

[1730] A "server" is a computer system that processes various data, such as analyzing user data and emotional data, generating advertisements, and providing information.

[1731] "Local trends" refers to information that shows trends and patterns regarding user interests, popular events, facilities, etc. within a particular geographic area.

[1732] "Generative AI models" refer to artificial intelligence algorithms and learning models used to generate personalized ads and location information using user and emotional data as input.

[1733] "Personalized advertising or location information" refers to advertising or location information that is individually optimized based on a user's individual interests, preferences, behavioral history, and emotional state.

[1734] "User terminal" refers to a communication device used by a user, such as a smartphone or tablet.

[1735] "Display device" refers to a device, such as an electronic bulletin board or digital signage, that visually presents advertisements or information to users.

[1736] "Push notification" is a messaging service that sends information from a server to a user's device in real time.

[1737] "Interact" means that a user interacts in some way with the advertisement or location information provided.

[1738] This invention is a system that collects and analyzes user location information, interests, preferences, behavioral data, and emotional data, and provides personalized advertisements and local information in real time. This system operates mainly using the following hardware and software.

[1739] Hardware

[1740] 1. User device: a smartphone or tablet, including a GPS and a camera.

[1741] 2. Display devices: digital signage and electronic bulletin boards.

[1742] software

[1743] 1. Emotion Engine: Facial expression recognition and voice analysis software to capture the user's emotional state.

[1744] 2. Generative AI model: Runs on the server and generates personalized ads and local information based on user data and emotional data.

[1745] 3. Data transmission / reception module: A communication module for exchanging data between the user terminal and the server.

[1746] Process Overview

[1747] 1. Acquisition and transmission of user data

[1748] When a user launches a smartphone app, the device's GPS function is used to obtain current location information. The app also collects the user's interests, preferences, and behavioral history. This data is sent to a server in real time.

[1749] 2. Acquiring Emotion Data

[1750] The smartphone is equipped with an emotion engine that analyzes the user's facial expressions and tone of voice, and this acquires emotional data, which is also sent to the server.

[1751] 3. Data analysis and ad generation

[1752] On the server side, user and sentiment data is analyzed to understand specific local trends and user interests, which is then used by generative AI models to generate personalized ads and local information.

[1753] 4. Provision of Information

[1754] The generated advertisements and local information are sent in real time to the user's device or display device, and on smartphones, the information is provided via push notifications.

[1755] Specific examples

[1756] For example, if a user is in a shopping mall, the generative AI model can generate special sale information and coupons for the mall based on the smartphone's location information, collected interest and preference data, and the relaxed state detected by the emotion engine, and provide them to the user via push notification.

[1757] Prompt Sentence Examples

[1758] "Use up-to-date sentiment data to generate ads that are best suited to relaxed users in a specific location."

[1759] This will enable us to provide users with the information they need in a timely manner, improving the effectiveness of our advertising. In addition, by collecting user feedback data, we will be able to further improve the accuracy of the information we provide next time.

[1760] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1761] Step 1:

[1762] When a user launches a smartphone app, the device uses its GPS function to obtain the user's current location. This location information is input data that the application sends to the server as user data. This location information is used to inform the server of the user's current location. Specifically, the smartphone's GPS sensor obtains the location information, and the application sends it to the server via an HTTP request.

[1763] Step 2:

[1764] The user's interests, preferences, and past behavioral data are also collected at the same time. This information, such as the user's past search keywords and visited places, is sent to the server along with the interest and preference data entered within the app. This input data forms an individual user profile and is used by the server to analyze the user's interests. Specifically, past history data is collected from the application's database and sent to the server along with location information.

[1765] Step 3:

[1766] The device is equipped with an emotion engine that analyzes the user's facial expressions (using the camera) and voice tone (using the microphone) to obtain emotion data. This emotion data is also sent to the server. The emotion data as input data indicates the user's current emotional state and is used to personalize advertisements. Specifically, the device uses the smartphone's camera and microphone to collect the user's biometric data, and the results analyzed by the emotion engine are sent to the server.

[1767] Step 4:

[1768] The server analyzes the received location information, interest and preference data, behavioral data, and emotional data. For this analysis, it uses big data analytics technology to gain insights based on local trends and user interests. This process includes data processing and calculation, such as data retrieval from the database, cleansing, point analysis, and trend analysis. Specifically, it retrieves trend data from the database and extracts information on popular events and stores by region.

[1769] Step 5:

[1770] The server inputs the analysis results into a generative AI model to generate personalized ads or local information based on the user's current location and emotional state. In this generation process, the generative AI model generates the optimal ads or information based on the input data and generates the results. Specifically, the generative AI model matches the user's profile with trend data to generate the optimal ads or information. An example of a prompt is, "Use the latest emotional data to generate the optimal ads for relaxed users in a specific area."

[1771] Step 6:

[1772] The generated advertisements and local information are sent from the server to the user's device or display device. In the case of smartphones, the information is provided to the user via push notification. The generated results as input data are sent to the device as an HTTP response, and the device displays the push notification. Specifically, the server returns the generated results as an HTTP response, and the smartphone receives the data and triggers the push notification.

[1773] Step 7:

[1774] When a user interacts with the generated advertisement or local information, the behavioral data is recorded and sent to the server. This behavioral data includes the click history of push notifications and the time spent viewing advertisements. Specifically, the system captures user operation events and sends them to the server as behavioral data.

[1775] Step 8:

[1776] The server analyzes the interaction data and uses it to generate the next personalized advertisements and local information. This analysis process includes analyzing past behavioral patterns and evaluating emotional data. Specifically, the behavioral history stored in the database is analyzed and used as input data for the next generation AI model.

[1777] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1778] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1779] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1780] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1781] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1782] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1783] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1784] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1785] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1786] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1787] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1788] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1789] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1790] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1791] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1792] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1793] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1794] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1795] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1796] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1797] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1798] The following is further disclosed regarding the above embodiment.

[1799] (Claim 1)

[1800] A means for obtaining user location information;

[1801] means for transmitting the acquired user data including the location information to a server;

[1802] A means for the server to analyze regional trends based on user data;

[1803] A means for inputting the analysis results into a generative model to generate personalized advertisements or local information;

[1804] The system includes means for transmitting the generated advertisement or local information to a user terminal or electronic billboard.

[1805] (Claim 2)

[1806] 10. The system of claim 1, further comprising means for recording user behavior as they interact with the generated advertisement or local information and transmitting the behavior data to the server.

[1807] (Claim 3)

[1808] 10. The system of claim 1, further comprising means for the server to analyze the interaction data and utilize it in generating subsequent personalized advertisements or location information.

[1809] "Example 1"

[1810] (Claim 1)

[1811] A means for obtaining user location information;

[1812] means for transmitting the acquired user data, including location information and interest and preference data, to a server;

[1813] A means for transmitting user behavior data from the device to a server;

[1814] A means for the server to analyze regional trends based on user data and external data using a big data analysis tool;

[1815] A means for inputting the analysis results into a generative AI model to generate personalized advertisements or local information;

[1816] The system includes means for transmitting the generated advertisement or location information to a user terminal or display device.

[1817] (Claim 2)

[1818] 10. The system of claim 1, further comprising means for recording user behavior as they interact with the generated advertisement or local information and transmitting the behavior data to the server.

[1819] (Claim 3)

[1820] 10. The system of claim 1, further comprising means for the server to analyze the interaction data and utilize it in generating subsequent personalized advertisements or location information.

[1821] "Application Example 1"

[1822] (Claim 1)

[1823] means for obtaining location information of an autonomous vehicle;

[1824] ...

Claims

1. A means for obtaining user location information; means for transmitting the acquired user data including the location information to a server; A means for the server to analyze regional trends based on user data; A means for inputting the analysis results into a generative model to generate personalized advertisements or local information; The system includes means for transmitting the generated advertisement or local information to a user terminal or electronic billboard.

2. The system of claim 1 , further comprising means for recording user behavior as they interact with the generated advertisement or local information and transmitting the behavior data to a server.

3. The system of claim 1 , further comprising means for the server to analyze the interaction data and use it to generate subsequent personalized advertisements or local information.

Images