system

Abstract

A system is provided.SOLUTION: The system includes a means for collecting and analyzing news data from a plurality of news sources in real time using a generative AI to extract important information, a means for generating an XR image based on the collected information, and a means for transmitting the generated XR image to a user terminal and causing the user to view the XR image, collects data related to online behavior and preference of the user, analyzes the collected data using the generative AI, generates an advertisement optimized for each user, and transmits the generated customized advertisement to the user terminal to be displayed to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern society demands rapid and accurate information provision in times of aging and disasters, but there is a lack of appropriate systems to meet this demand. Furthermore, information overload, fake news, and untargeted advertising make it difficult for users to obtain useful information. Advertisers also have limited means to effectively reach their target audiences, reducing the effectiveness of their advertising. This invention aims to solve these issues by combining generative AI and XR video technology. [Means for solving the problem]

[0005] A system that improves the accuracy and effectiveness of information provision and advertising distribution by providing: a means for using a generating AI to collect and analyze news data from multiple news sources in real time and extract important information; a means for generating XR video based on the extracted information; a means for transmitting the generated XR video to a user terminal and allowing the user to view it; a means for collecting live sports data and extracting important highlights and key points of the game using a generating AI; a means for generating a virtually reproduced sports game video based on the extracted data; a means for transmitting the generated virtual sports video to a user terminal and allowing the user to view it; a means for collecting data on users' online behavior and preferences; a means for using a generating AI to analyze the collected data and generate advertisements optimized for individual users; and a means for transmitting the generated customized advertisements to a user terminal and displaying them to the user.

[0006] "Generative AI" is a system that uses artificial intelligence techniques to analyze data and generate new content.

[0007] "News Source" refers to multiple sources of news data, including APIs, RSS feeds, and news websites.

[0008] "Real-time" means instantly processing and providing information about events occurring near the present time.

[0009] "News data" refers to information data collected from news sources, including articles, images, videos, etc.

[0010] "Important information" refers to information extracted from news data that is particularly valuable to users.

[0011] "XR video" refers to video generated using augmented reality technology, and includes content that combines the real world with virtual information.

[0012] "User terminal" refers to an electronic device used by a user to receive and view information, and includes smartphones, tablets, head-mounted displays, etc.

[0013] "Live sports data" refers to data about sporting events provided in real time, including scores, player movements, and game progress.

[0014] "Key highlights" refer to important scenes or events in a match that deserve particular attention.

[0015] "Virtually recreated sports game footage" refers to footage of a virtual game generated based on real-time sports data.

[0016] "Online behavior" refers to all activities a user engages in on the Internet, including website visits, clicks, and purchases.

[0017] "Customized advertising" refers to individually optimized advertising generated based on a user's interests and behavioral patterns. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI, and also provides customized advertisements based on the user's online behavior and preferences. The specific operation form and program processing of this system are explained below.

[0040] Providing real-time news

[0041] The server collects news data from multiple news sources (APIs, RSS feeds, news websites, etc.). The server then uses generative AI to analyze the collected data and extract key information. The server generates XR video based on the extracted information and sends the data to the user's device. The device visualizes the received XR video data, and the user watches the XR video.

[0042] Specific examples

[0043] 1. The server retrieves news data through the news API.

[0044] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[0045] 3. The server uses tools such as Unity or Unreal Engine to generate important information as XR images.

[0046] 4. The server sends the generated XR video data to the user's device, which then displays the data, allowing the user to watch the news in an immersive manner.

[0047] Providing virtual sports experiences

[0048] The server collects live sports data (APIs, sports websites, real-time data feeds, etc.). The server then uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. The server generates a virtual sports game video based on the extracted data and sends the data to the user's device. The device visualizes the received virtual sports video, allowing the user to watch the game.

[0049] Specific examples

[0050] 1. The server retrieves live match data through the sports API.

[0051] 2. The server analyzes the match using a generated AI and extracts important highlights.

[0052] 3. The server uses 3D modeling tools to virtually generate footage of the match.

[0053] 4. The server sends the generated virtual sports video data to the user's device, which then displays this data, allowing the user to experience an immersive game.

[0054] Providing customized advertising

[0055] The server collects data about a user's online behavior and preferences from cookies and web traffic. The server then uses generative AI to analyze the collected data and generate ads optimized for individual users. The server then sends the customized ads it generates to the user's device, which displays them. The user views the ads and, if interested, accesses a page for additional information or a purchase.

[0056] Specific examples

[0057] 1. The server uses web analysis tools to collect user behavior data.

[0058] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[0059] 3. The server then generates a personalized ad based on this information.

[0060] 4. The server sends the generated advertising data to the user's device, which displays the advertisement, allowing the user to access the advertisement and, if interested, go directly to a page for more information or to purchase.

[0061] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements, thereby realizing fast and accurate information provision and improved advertising effectiveness in an aging society or during disasters.

[0062] The processing flow will be explained below.

[0063] Providing real-time news

[0064] Step 1:

[0065] The server collects the latest news data from multiple news sources (e.g., news APIs, RSS feeds, news websites).

[0066] Step 2:

[0067] The server uses generated AI to analyze the collected news data and extract important keywords and events, specifically using natural language processing (NLP) algorithms.

[0068] Step 3:

[0069] Based on the key information extracted by the server, XR images (e.g., 3D models and infographics) are generated using engines such as Unity or Unreal Engine.

[0070] Step 4:

[0071] The server sends the generated XR video data to the user's device.

[0072] Step 5:

[0073] The device displays the received XR video data, allowing users to watch and experience the news in an immersive format.

[0074] Providing virtual sports experiences

[0075] Step 1:

[0076] A server collects live sports data (e.g., sports API, real-time data feeds).

[0077] Step 2:

[0078] The server uses generative AI to analyze the collected sports data and extract important highlights and key points from the match, such as scores, player movements, and interesting scenes.

[0079] Step 3:

[0080] The server generates a virtually recreated sports game video based on the extracted data, again using engines such as Unity and Unreal Engine.

[0081] Step 4:

[0082] The server transmits the generated virtual sports video data to the user's terminal.

[0083] Step 5:

[0084] The device displays the received virtual sports video data, allowing users to watch it and experience it as if they were watching a real game.

[0085] Providing customized advertising

[0086] Step 1:

[0087] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[0088] Step 2:

[0089] The server uses generated AI to analyze the collected data and identify individual users' interests and behavioral patterns, specifically by analyzing their browsing history and click patterns.

[0090] Step 3:

[0091] Based on the analysis results, the server generates individually optimized advertisements, which are customized to the user's interests.

[0092] Step 4:

[0093] The server transmits the generated customized advertisement to the user's terminal.

[0094] Step 5:

[0095] The device displays the received advertisement, and if the user is interested, they can use a QR code or clickable element to access more information or a purchase page.

[0096] Through these processing steps, users can efficiently receive important news, virtual sports experiences, and customized advertising in real time.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] There is a growing need for fast and accurate information provision in an aging society and during disasters. There is also a demand for customized advertising that meets individual user needs. Conventional methods are difficult to effectively address these challenges, so a system that can provide highly accurate information in real time and a customized experience is needed.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes means for collecting data in real time from multiple information sources using a generation AI and analyzing and extracting important information, means for generating mixed reality video based on the extracted information, and means for transmitting the generated mixed reality video to a user terminal and allowing the user to view it. This allows important information to be provided quickly in real time in an immersive format, making it possible to provide information in an aging society or during disasters.

[0102] "Generative AI" is an artificial intelligence technology that uses techniques such as machine learning and deep learning to generate and analyze data and extract specific information.

[0103] "Source" refers to the original media or platform that provides the data, such as an API, RSS feed, news website, or sports data feed.

[0104] "Real-time" refers to a situation in which data is generated, acquired, and analyzed almost simultaneously with real time, with extremely little delay.

[0105] "Data collection" is the process by which the server obtains data from external sources and stores it in its database.

[0106] "Analysis" is the process of processing collected data using generative AI to extract important information.

[0107] "Important information" refers to keywords, events, or phenomena that are particularly noteworthy among the collected data.

[0108] "Mixed reality video" refers to video content that combines the real world and the virtual world, generated using virtual reality (VR) or augmented reality (AR) technology.

[0109] "User device" refers to electronic devices used by users, such as computers, smartphones, tablets, and VR headsets.

[0110] "To allow a user to view" refers to the act of making a video or content available for a user to view through a user terminal.

[0111] "Live event data" refers to data that includes information about an event that is unfolding in real time, such as a sports game or a concert.

[0112] A "highlight" is a particularly important or noticeable part of an event or piece of data.

[0113] "Virtually recreated event footage" refers to footage that is virtually generated based on an actual event using 3D modeling and animation.

[0114] "Online behavior" refers to all the activities a user engages in on the Internet, including browsing, clicking, and purchasing history.

[0115] "Customized advertising" refers to advertising generated based on a user's individual interests and behavioral patterns.

[0116] This invention is a system that uses generative AI to collect data from multiple sources in real time, extract and analyze important information, and provide it to users. The system of the present invention provides visual mixed reality images to the user terminal.

[0117] Specific system configuration

[0118] Data collection

[0119] A server collects news and live event data in real time from multiple sources (APIs, RSS feeds, news websites, etc.) In this process, the server periodically sends HTTP requests to API endpoints and receives data in JSON format.

[0120] Data analysis and key information extraction

[0121] The server analyzes the collected data using a generative AI model. This analysis automatically extracts important keywords and events. For example, by providing the generative AI model with a prompt such as "Analyze the news data and extract important information," the server obtains the AI's analysis results.

[0122] Mixed reality image generation

[0123] The server uses the extracted information to generate mixed reality images using 3D modeling tools such as Unity or Unreal Engine. For example, a prompt such as "simulate a news event" can be sent to Unity, and a 3D scene can be constructed based on the simulation results.

[0124] Video data transmission

[0125] The server sends the generated mixed reality video data to the user's device using a network protocol (e.g., WebSocket or HTTP / 2) to stream the video data with low latency.

[0126] Visualization of video data

[0127] The device visualizes the received mixed reality video data and allows the user to view it. A dedicated application runs on the device, decodes the received data, and displays it on a VR headset or AR display.

[0128] Specific examples

[0129] News data provision

[0130] 1. The server retrieves news data through the news API from a URL such as "https: / / newsapi.org / v2 / top-headlines?country=jp&apiKey=YOUR_API_KEY".

[0131] 2. The server uses the generated AI to analyze the data with the prompt, "Analyze the news data and extract important information."

[0132] 3. The server uses Unity to generate a 3D scene based on a "simulated news event."

[0133] 4. The server sends the generated XR video data to the user's device.

[0134] 5. The device visualizes the received data, and the user watches an immersive news video on a VR headset.

[0135] Providing virtual sports experiences

[0136] 1. The server retrieves live match data through the sports API.

[0137] 2. The server uses a generative AI to analyze the data with the prompt, "Extract key highlights from the match data."

[0138] 3. The server uses a 3D modeling tool to virtually generate a "reenactment of the goal scene."

[0139] 4. The server sends the generated virtual sports video data to the user's device.

[0140] 5. The device visualizes the received data, allowing the user to watch an immersive game video.

[0141] Providing customized advertising

[0142] 1. The server uses web analysis tools to collect user behavior data.

[0143] 2. The server uses the generation AI to analyze the data with the prompt, "Analyze user behavior data and generate ads based on it."

[0144] 3. The server uses generated AI to generate individually customized ads.

[0145] 4. The server sends the generated advertisement to the user's device.

[0146] 5. The device displays the received advertisement, and if the user is interested, they can access the page for more information or to purchase.

[0147] In this way, the present invention provides a system that allows for fast and accurate information delivery and a customized experience.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Providing real-time news

[0150] Processing Steps

[0151] Step 1:

[0152] The server collects news data from multiple sources (APIs, RSS feeds, news websites, etc.). Specifically, the server periodically sends HTTP requests to API endpoints to retrieve data in JSON format. (Input) News API endpoint URL, (Output) JSON-formatted news data.

[0153] Step 2:

[0154] The server uses a generative AI model to analyze the collected news data. It uses the collected JSON data as input and provides the prompt, "Analyze the news data and extract important information," to the generative AI model, which then extracts important keywords and event information. (Input) JSON-formatted news data, prompt, (Output) Analysis results including important keywords and event information.

[0155] Step 3:

[0156] The server uses 3D modeling tools such as Unity or Unreal Engine to generate XR video based on the analysis results. The 3D modeling tool is given a prompt statement, "simulation of a news event," and a 3D scene is constructed based on key information. (Input) Analysis results, prompt statement, (Output) Generated XR video data.

[0157] Step 4:

[0158] The server sends the generated XR video data to the user's device. The server uses network protocols such as WebSocket or HTTP / 2 to stream the data with low latency. (Input) Generated XR video data. (Output) Transmission to the user's device is complete.

[0159] Step 5:

[0160] The device visualizes the received XR video data and the user views it. A dedicated application on the device decodes the data and displays the immersive video on a VR headset or AR display. (Input) Received XR video data, (Output) Visualized video (viewed by the user).

[0161] Providing virtual sports experiences

[0162] Processing Steps

[0163] Step 1:

[0164] The server uses the Sports API to collect live match data. The server sends a request to the API endpoint to retrieve data such as match scores and player movements in JSON format. (Input) Sports API endpoint URL, (Output) Live match data in JSON format.

[0165] Step 2:

[0166] The server uses a generative AI model to analyze live match data and extract important highlights. The generative AI model is given a prompt statement, "Please extract key highlights from the match data." The input is the live match data in JSON format, the prompt statement, and the output is the analysis results including important highlight information.

[0167] Step 3:

[0168] The server uses a 3D modeling tool to generate virtual sports footage based on the extracted highlight information. The 3D modeling tool is given the prompt "Reproduce the goal scene" to recreate the game scene. (Input) Highlight information, prompt, (Output) Generated virtual sports footage data.

[0169] Step 4:

[0170] The server sends the generated virtual sports video data to the user device. The server uses WebSocket to stream the data to the user device with low latency. (Input) Generated virtual sports video data, (Output) Transmission to the user device completed.

[0171] Step 5:

[0172] The device visualizes the received virtual sports video data and the user watches it. A dedicated application decodes the data, allowing the user to watch the game immersively using a VR headset. (Input) Received virtual sports video data, (Output) Visualized video (user viewing).

[0173] Providing customized advertising

[0174] Processing Steps

[0175] Step 1:

[0176] The server uses web analysis tools to collect data on users' online behavior. It collects cookies and web traffic data and logs users' browsing history and click behavior. (Input) User access logs and cookies, (Output) User online behavior data.

[0177] Step 2:

[0178] The server uses a generative AI model to analyze the behavioral data and determine the user's interests and behavioral patterns. It then provides the generative AI model with a prompt: "Analyze the user's behavioral data and generate advertisements based on it." (Input) The user's online behavioral data, the prompt, and (Output) the analysis results, including appropriate customized advertisement information.

[0179] Step 3:

[0180] The server uses the generative AI model to generate individually customized ads. It generates ads based on the analysis results and designs advertising campaigns. (Input) Analysis results, prompt text, (Output) Generated customized ads.

[0181] Step 4:

[0182] The server sends the generated customized advertisement to the user terminal. The server sends the advertisement data to the user terminal using the HTTP protocol. (Input) Generated customized advertisement, (Output) Transmission to the user terminal completed.

[0183] Step 5:

[0184] The device displays the received customized advertisement and the user views it. The advertisement is displayed in a browser or an application dedicated to displaying advertisements, allowing the user to access advertisements that interest them. (Input) Received customized advertisement, (Output) Displayed advertisement (viewed by the user).

[0185] (Application example 1)

[0186] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0187] There is a need for systems that can provide fast and accurate information in an aging society or during disasters. There is also a need for effective means of providing information so that security guards can patrol efficiently. While satisfying these requirements, there is also a need for systems that can display customized advertisements according to user preferences.

[0188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0189] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing security-related information using the generation AI and customizing and providing optimal patrol routes and warning information, and means for displaying the customized information on the smart glasses, thereby enabling prompt and accurate information provision and efficient security management.

[0190] "Generative AI" is a system that uses artificial intelligence techniques to generate, analyze, and learn from data.

[0191] "News source" refers to an API, RSS feed, news website, or other source that provides news data.

[0192] "News Data" refers to articles, reports, footage, and other news-related information collected from news sources.

[0193] "Important information" refers to information such as key points, important keywords, and events extracted from news data.

[0194] "XR video" refers to video generated using technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0195] "User terminal" refers to a device used by a user, such as a smartphone, tablet, smart glasses, or head-mounted display.

[0196] "Security-related information" refers to patrol routes, warnings, alarm information, etc. required by security guards.

[0197] A "patrol route" refers to the optimal route for a security guard to patrol.

[0198] "Warning information" refers to data that provides information on where particular caution should be exercised in a particular situation or location.

[0199] "Smart glasses" refers to a glasses-type device equipped with a display for displaying information.

[0200] "Customized advertising" refers to advertising content that is optimized based on a user's online behavior and preferences.

[0201] In this invention, the following system configuration is adopted to realize an application for smart glasses specialized for security services.

[0202] First, the server collects news data in real time from news sources, such as news APIs, RSS feeds, and news websites. Then, the server uses generative AI to analyze the collected news data and extract important information. A possible generative AI model for this purpose is OpenAI GPT-4.

[0203] Based on the extracted important information, the server generates XR images. It is recommended to use a 3D modeling tool such as Unity or Unreal Engine. The generated XR images are then sent to the user's device, such as smart glasses.

[0204] The server also analyzes security-related information in real time. It uses AI to analyze patrol routes and warning information and provides customized information. This information is also displayed on the smart glasses. This allows security guards to quickly obtain information on optimal patrol routes and important points to pay attention to.

[0205] In addition, the server collects and analyzes users' online behavior data and past patrol data. It uses generative AI to analyze this data and generate advertising information optimized for each individual user. The generated advertising information is then sent to the user's device and displayed on the smart glasses.

[0206] Specific examples

[0207] Security guard A wears smart glasses while on duty. The glasses display the latest earthquake information and evacuation instructions in real time as XR video. They also suggest optimal patrol routes based on past patrol data analyzed by the server. Additionally, the glasses display advertisements customized based on A's web browsing history.

[0208] Prompt Sentence Examples

[0209] Extract important information from the following news data: {News Data}

[0210] This invention allows security guards to take action quickly and accurately, realizing efficient security management. In addition, by displaying advertisements according to user preferences, it is expected that the effectiveness of advertisements will be improved.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] The server collects news data from news sources, specifically, news APIs, RSS feeds, news websites, etc., via API requests. The input is the URL or API endpoint of the news source, and the output is the data of the news article or report.

[0214] Step 2:

[0215] The server analyzes the collected news data using generative AI to extract important information. The input is the news data collected in the previous step, and important keywords and events are extracted by sending prompts to the generative AI model. The output is the extracted important information, which is used in subsequent steps.

[0216] Step 3:

[0217] The server generates XR video based on the extracted key information. For this purpose, it uses 3D modeling tools such as Unity or Unreal Engine. The input is the key information extracted in step 2, and the output is the generated XR video data.

[0218] Step 4:

[0219] The server sends the generated XR video data to the user device. The input is the XR video data generated in step 3, and also includes the address of the destination user device. The output is a message that the data was successfully sent to the user device.

[0220] Step 5:

[0221] The user device visualizes the XR video data received from the server and displays it to the user. The input is the XR video data sent from the server, and the output is the video displayed on a display such as smart glasses.

[0222] Step 6:

[0223] The server collects security-related information and analyzes it using generative AI. The input is security data (e.g., security logs, surveillance camera feeds), and the analysis extracts optimal patrol routes and warning information. The output is customized security information.

[0224] Step 7:

[0225] The server sends customized patrol routes and warning information to the smart glasses. The input is the security information generated in step 6, and the output is a data transmission success message to the smart glasses.

[0226] Step 8:

[0227] The smart glasses visualize customized patrol routes and warning information received from the server and display it to the user. The input is the security information sent from the server, and the output is the information displayed on the smart glasses' display.

[0228] Step 9:

[0229] The server collects the user's online behavior data and analyzes it using Generative AI. The input is the user's online behavior data (e.g., web traffic, cookies) and analyzes the user's interests and behavioral patterns. The output is advertising information optimized for each individual user.

[0230] Step 10:

[0231] The server sends the generated customized advertisement to the user terminal, where the input is the advertisement information generated in step 9, and the output is a data transmission success message to the user terminal.

[0232] Step 11:

[0233] The user terminal displays the customized advertisement received from the server. The input is the advertisement data sent from the server, and the output is the advertisement video that the user can view.

[0234] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0235] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI and an emotion engine, and also provides customized advertisements based on the user's online behavior and emotions. The specific operation form and program processing of this system are explained below.

[0236] Providing real-time news

[0237] The server collects news data from multiple news sources (e.g., news APIs, RSS feeds, and news websites). The server then uses generative AI to analyze the collected news data and extract important keywords and events. The server then uses an emotion engine to recognize the user's emotions and customizes the XR video of the news data based on the user's emotions. This ensures that when the video is sent to the user's device, it contains content that corresponds to the user's specific emotions. The device displays this customized XR video data, allowing the user to watch the news.

[0238] Specific examples

[0239] 1. The server retrieves news data through the news API.

[0240] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[0241] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the news content.

[0242] 4. The server generates customized XR footage using tools such as Unity or Unreal Engine.

[0243] 5. The server sends the customized XR video data generated to the user's device, which then displays it, allowing the user to watch news content tailored to their emotions in an immersive format.

[0244] Providing virtual sports experiences

[0245] The server collects live sports data (e.g., sports APIs, real-time data feeds). Then, the server uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. Furthermore, the server uses an emotion engine to recognize the user's emotions and customize the virtual sports game video based on the user's emotions. The device displays this customized virtual sports video for the user to watch.

[0246] Specific examples

[0247] 1. The server retrieves live match data through the sports API.

[0248] 2. The server analyzes the match using a generated AI and extracts important highlights.

[0249] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the content of the game footage.

[0250] 4. The server uses 3D modeling tools to generate a customized virtual match.

[0251] 5. The server sends the generated customized virtual sports video data to the user's device, which then displays it, allowing the user to experience a match that matches their emotions in an immersive format.

[0252] Providing customized advertising

[0253] The server collects users' online behavior data from cookies and web traffic. The server then uses generative AI to analyze the collected data and identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions and further customize the advertising content. The server then sends the customized advertisements it generates to the user's device, which displays them. The user views the advertisements and, if interested, accesses a page for additional information or to purchase.

[0254] Specific examples

[0255] 1. The server uses web analysis tools to collect user behavior data.

[0256] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[0257] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the advertising content.

[0258] 4. The server then generates a personalized ad based on this information.

[0259] 5. The server sends the generated advertising data to the user's device, which displays it, allowing the user to access an advertisement with content that corresponds to their emotions, and if they are interested, they will be taken directly to a page with more information or to a purchase page.

[0260] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements based on their emotions, which will enable the provision of fast and accurate information in an aging society or during disasters, as well as further improving advertising effectiveness.

[0261] The processing flow will be explained below.

[0262] Providing real-time news

[0263] Step 1:

[0264] The server collects news data in real time from multiple news sources such as news APIs, RSS feeds, and news websites.

[0265] Step 2:

[0266] The news data collected by the server is analyzed using generative AI to identify important keywords and events.

[0267] Step 3:

[0268] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[0269] Step 4:

[0270] The server customizes news content based on the user's emotional data, adjusting the tone and content to match specific emotions.

[0271] Step 5:

[0272] The server generates customized news data as XR video using tools such as Unity and Unreal Engine.

[0273] Step 6:

[0274] The server sends the generated customized XR video data to the user's device.

[0275] Step 7:

[0276] The device displays the received customized XR video data, and the user watches it to experience customized news tailored to their emotions in an immersive format.

[0277] Providing virtual sports experiences

[0278] Step 1:

[0279] The server collects live sports data from sports APIs, real-time data feeds, etc.

[0280] Step 2:

[0281] The server uses generated AI to analyze the collected sports data and extract important highlights and key points from the game.

[0282] Step 3:

[0283] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[0284] Step 4:

[0285] The server customizes the game footage based on the user's emotional data, adjusting visual effects and narration to match specific emotions.

[0286] Step 5:

[0287] The server generates customized virtual sports game footage using 3D modeling tools.

[0288] Step 6:

[0289] The server transmits the generated customized virtual sports video data to the user's terminal.

[0290] Step 7:

[0291] The terminal displays the received customized virtual sports video data, and the user watches the video data to experience a customized match according to their emotions in an immersive format.

[0292] Providing customized advertising

[0293] Step 1:

[0294] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[0295] Step 2:

[0296] The server uses generated AI to analyze the collected behavioral data and identify the user's interests and behavioral patterns.

[0297] Step 3:

[0298] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[0299] Step 4:

[0300] The server further customizes the optimized advertising content based on the user's emotional data, generating advertising content that corresponds to a specific emotion.

[0301] Step 5:

[0302] The server generates customized advertising data, with specific designs and messages tailored to the emotion.

[0303] Step 6:

[0304] The server transmits the generated customized advertisement data to the user's terminal.

[0305] Step 7:

[0306] The device displays the personalized ad it receives, the user watches the ad, engages with the ad with emotional content, and if interested, is taken directly to a page with more information or a purchase using a QR code or clickable element.

[0307] These processing steps effectively allow users to receive important news, virtual sports experiences, and customized advertising in real time, while the incorporation of an emotion engine provides a more personalized experience.

[0308] Example 2

[0309] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] In modern society, information overload, misinformation, biased reporting, and other factors make it difficult for users to quickly and accurately obtain the information they need. Furthermore, in an aging society or during disasters, providing information to individual users becomes increasingly important, but general media services have difficulty providing appropriate information based on the user's emotions and state. Furthermore, current advertising systems are one-sided and are unable to provide effective advertising that is in line with the user's emotions and interests.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0312] In this invention, the server includes means for collecting and analyzing data from multiple information sources in real time using a generation AI and extracting important information, means for recognizing a user's emotions using an emotion engine, and means for generating XR video based on the extracted information and the recognized emotions. This enables users to receive appropriate information according to their emotions quickly and in an immersive format, which is expected to improve information provision in an aging society or during disasters, as well as further increase the effectiveness of advertising.

[0313] "Generative AI" refers to algorithms and models that use artificial intelligence technology to generate data such as text and images.

[0314] "Source" refers to the means by which information is obtained from digital sources, such as news APIs, RSS feeds, websites, etc.

[0315] "Real-time" means that data is processed and transmitted the instant it is generated or acquired.

[0316] "Emotion engine" refers to algorithms and models for analyzing and recognizing user emotions.

[0317] "XR video" refers to video formats including augmented reality (AR), virtual reality (VR), and mixed reality (MR).

[0318] "User terminal" refers to digital devices such as smartphones, tablets, and personal computers.

[0319] "Live Data" refers to dynamic data that is generated and captured in real time.

[0320] "Online behavioral data" refers to data about the history and patterns of a user's activities on the Internet.

[0321] "Customized advertising" refers to advertising that is optimized based on the interests and emotions of individual users.

[0322] This invention is a comprehensive media system that uses generative AI and an emotion engine to provide users with customized news, virtual sports experiences, and advertising. Specific embodiments of this system are described below.

[0323] News data provision

[0324] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to obtain news data in real time. The collected data undergoes text analysis using generative AI (e.g., GPT-4) to extract important keywords and events. The server then uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Based on the user's emotions, the server adjusts and customizes the content of the news data. The server then generates customized XR video using tools such as Unity or Unreal Engine. The generated XR video data is sent to the user's device, where the user can view it.

[0325] As a specific example, we will adopt a method of collecting the latest earthquake news and displaying information that will reassure users who are feeling anxious. An example of a prompt sentence would be, "Collect the latest earthquake news and display it in a format that will reassure users who are feeling anxious."

[0326] Providing virtual sports experiences

[0327] The server accesses sports APIs and real-time data feeds to obtain live sports data. The collected data is analyzed by generative AI to extract important highlights and key points from the match. The server then uses an emotion engine to recognize the user's emotions in real time. Based on this, the content of the match video is adjusted and customized. Specifically, it provides more thrilling footage to excited users. The server uses 3D modeling tools such as Blender and Maya to generate customized virtual match video. The generated video data is sent to the user's device, where the user can watch it.

[0328] As a concrete example, live soccer data is analyzed and videos that highlight goal scenes, fine plays, etc. An example of a prompt sentence is "Analyze live soccer data and generate exciting videos for excited users."

[0329] Providing customized advertising

[0330] The server collects online behavior data from cookies and web traffic. The collected data is analyzed by Generative AI to identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions in real time. Based on this, advertising content is adjusted and customized. Specifically, an excited user will be served an energetic advertisement. The server then uses Generative AI to generate a customized advertisement and sends this data to the user's device. The user views the advertisement on their device and, if interested, accesses a page for additional information or to purchase.

[0331] As a concrete example, a user's online behavior data is analyzed, and if the user is interested in a particular product, an advertisement mainly consisting of positive reviews and videos about that product is provided. An example of a prompt sentence would be, "Analyze the user's online behavior data and generate a positive advertisement for the product they are interested in."

[0332] This system allows users to receive emotionally appropriate information quickly and in an immersive format, which is expected to improve information provision in aging societies and during disasters, and further increase the effectiveness of advertising.

[0333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0334] News data provision

[0335] Step 1: Gathering news data

[0336] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to collect news data in real time. Specifically, it obtains data in JSON format through a REST API. The input of this step is news data obtained from the sources, and the output is the collected, unparsed news data.

[0337] Step 2: Analyzing news data and extracting keywords

[0338] The server uses a generative AI (e.g., GPT-4) to analyze the collected news data. It uses a text analysis algorithm to extract important keywords and events. The input of this step is the collected news data, and the output is the extracted important keywords and event information.

[0339] Step 3: Recognizing user emotions

[0340] The server uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Specifically, it analyzes the user's past behavioral data and real-time sensor data to estimate the user's emotional state. The input for this step is the user's real-time and historical data, and the output is the user's emotional state.

[0341] Step 4: Customize your news content

[0342] The server adjusts and customizes the content of the news data based on the emotion recognition results. For example, it restructures the content by emphasizing information that gives a sense of security to a user who is feeling anxious. The inputs for this step are extracted keywords, event information, and the user's emotional state, and the output is customized news content.

[0343] Step 5: Generate XR footage

[0344] The server uses tools such as Unity or Unreal Engine to generate customized XR footage, specifically creating a 3D reproduction of the news and related visual effects. The input for this step is the customized news content, and the output is the generated XR footage.

[0345] Step 6: Send and view your customized XR footage

[0346] The server sends the generated customized XR video data to the user's device. The device displays the received video data, and the user watches it. The input to this step is the generated XR video data, and the output is the video displayed on the user's device.

[0347] Providing virtual sports experiences

[0348] Step 1: Collecting live sports data

[0349] The server accesses sports APIs and real-time data feeds to collect live sports data. Specifically, it retrieves data in JSON format through REST APIs. The input of this step is the sports data retrieved from the source, and the output is the collected, unparsed sports data.

[0350] Step 2: Analyzing sports data and extracting highlights

[0351] The server uses the generative AI to analyze the collected sports data. It uses text analysis and video analysis algorithms to extract important highlights and critical moments. The input of this step is the collected sports data, and the output is the extracted highlight information.

[0352] Step 3: Recognizing user emotions

[0353] The server uses an emotion engine to recognize the user's emotions in real time, using the user's past viewing history and real-time feedback data. The input of this step is the user's real-time and historical data, and the output is the user's emotional state.

[0354] Step 4: Customize your video content

[0355] The server adjusts and customizes the content of the game footage based on the emotion recognition results. For example, it edits the footage to be more thrilling for an excited user. The input of this step is the extracted highlight information and the user's emotional state, and the output is customized game footage.

[0356] Step 5: Generate virtual matches

[0357] The server generates customized virtual game footage using 3D modeling tools such as Blender or Maya. The input of this step is the customized game content, and the output is the generated virtual game footage data.

[0358] Step 6: Send and view your customized virtual game footage

[0359] The server sends the generated virtual game video data to the user terminal. The terminal displays the received video data, and the user watches it. The input to this step is the generated virtual game video data, and the output is the video displayed on the user terminal.

[0360] Providing customized advertising

[0361] Step 1: Collect online behavior data

[0362] The server collects online behavior data from cookies and web traffic. Specifically, it obtains the data using web analytics tools such as Google Analytics. The input of this step is the user's online behavior data, and the output is the collected raw data.

[0363] Step 2: Analyze interests and behavioral patterns

[0364] The server uses the generated AI to analyze the collected data. It uses an algorithm to identify individual user interests and behavioral patterns. The input of this step is the collected behavioral data, and the output is analyzed user interest and behavioral pattern information.

[0365] Step 3: Recognizing user emotions

[0366] The server uses an emotion engine to recognize the user's emotions in real time, using real-time data collected from the webcam and microphone. The input of this step is real-time data and historical data, and the output is the user's emotional state.

[0367] Step 4: Customize your ad content

[0368] The server adjusts and customizes the advertisement content based on the emotion recognition results. For example, if the user is excited, it will provide an energetic presentation. The input of this step is the analyzed interest information and emotional state, and the output is customized advertisement content.

[0369] Step 5: Generate a personalized ad

[0370] The server uses generative AI to generate individually optimized ads, adjusting visual effects and wording. The input for this step is the customized ad content, and the output is the generated ad data.

[0371] Step 6: Send and display personalized ads

[0372] The server sends the generated advertisement data to the user terminal. The terminal displays it, and the user watches the advertisement. The input of this step is the generated advertisement data, and the output is the advertisement displayed on the user terminal.

[0373] (Application example 2)

[0374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional news, sports video, and advertising systems lack personalization based on individual users' emotions and online behavior data, limiting the effectiveness of information provision. In particular, physical stores lack the means to provide users with appropriate product recommendations and promotional information, creating a demand for real-time, emotion-based customized information delivery. A system is needed to solve these issues and improve user experience.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0377] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing the user's emotional state, means for customizing the XR video based on the emotion analysis results, and means for providing personalized product recommendation information to the user in a store, thereby enabling the provision of customized information based on the user's emotions and online behavior data.

[0378] "Generative AI" is an artificial intelligence algorithm that generates and analyzes information from massive amounts of data.

[0379] "News source" refers to the online platform or information provider that provides news data.

[0380] "News data" refers to a collection of information reported as news, including text, images, and video.

[0381] "XR video" refers to augmented reality (AR), virtual reality (VR), and mixed reality (MR) video that combines the real world and the virtual world.

[0382] A "user terminal" is a device for receiving and displaying information, and includes smartphones, smart glasses, head-mounted displays, etc.

[0383] "Emotional state" refers to the user's current psychological feelings, such as "joy," "anger," or "sadness."

[0384] "Emotion analysis" refers to technology for recognizing and classifying a user's emotional state.

[0385] "Customization" refers to tailoring content and information to best suit individual users.

[0386] "Recommended information" refers to information about products and services that are suggested to users.

[0387] "Products" refers to goods and services sold in physical stores.

[0388] "In-store" refers to the physical location where goods and services are provided.

[0389] "Personalization" refers to the unique tailoring of information or services based on a user's individual attributes, behavior, or emotions.

[0390] "Promotional information" refers to information about sales, campaigns, events, etc. for sales promotion purposes.

[0391] This invention realizes a system that uses generative AI and an emotion engine to provide individually customized information and advertisements to users, which can improve the user's shopping experience, especially in physical stores.

[0392] System Overview:

[0393] This system uses generative AI to collect and analyze real-time news data, extract and generate important information, analyze the user's emotional state, and customize XR video and product recommendations based on the results. Finally, the customized information is sent to the user's device, allowing them to view it.

[0394] Hardware and software used:

[0395] Hardware: User devices such as smartphones, smart glasses, and head-mounted displays.

[0396] Software: Generative AI models, sentiment analysis engines, notification systems, 3D modeling tools such as Unity and Unreal Engine.

[0397] Processing Details:

[0398] 1. Data Collection:

[0399] The server collects news data in real time from multiple news sources, such as news APIs and RSS feeds, as well as users' online behavioral data and emotional states.

[0400] 2. Data Analysis:

[0401] Generative AI models are used to analyze collected news data and extract important keywords and events, as well as users' online behavior data, to identify the most relevant information for each individual user.

[0402] 3. Emotion analysis:

[0403] A sentiment analysis engine is used to analyze the user's current emotional state, which is then used to customize news and product recommendations.

[0404] 4. Information generation:

[0405] Based on the analysis results, a generative AI model is used to generate customized XR footage and recommendations, including 3D modeling using Unity or Unreal Engine.

[0406] 5. Information provision:

[0407] The generated customized information is sent to the user's device via the notification system, allowing the user to view news and product recommendations in real time according to their emotional state and interests.

[0408] Examples:

[0409] For example, consider a case where a user has a search history of "shoes" and "watches," has a history of purchasing "jeans," and is currently in a "happy" emotional state. Based on this data, the server uses an emotion analysis engine to analyze the user's emotions and generate recommendations such as "casual shoes" and "sports watches." At the same time, it notifies the user of current sales and campaign information.

[0410] Example prompt sentence:

[0411] "User ID 12345's current emotional state is 'happy', and his / her online behavior data includes 'shoes' and 'watches' in his / her search history and 'jeans' in his / her purchase history. Based on this, please generate recommended products and related promotion information for the user."

[0412] This system will enable users to receive information and advertisements in real time that are optimized to their emotions and behavior, enabling a more personalized shopping experience, especially in physical stores.

[0413] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0414] Step 1:

[0415] The server collects news data in real time from multiple news sources such as news APIs and RSS feeds. In this data collection process, it loads media data such as text, images, and videos from the news sources. The input is data from the news sources, and the output is a set of collected news data.

[0416] Step 2:

[0417] The server uses a generative AI model to analyze the news data collected and extract important keywords and events. Here, the input news data is analyzed using natural language processing technology to perform summarization and topic extraction. The output is the extracted important keywords and events.

[0418] Step 3:

[0419] The server collects the user's online behavior data and emotional state data. At this stage, it obtains the user's online behavior data, such as browsing history, search history, and purchase history, and also collects emotional state data from smart devices and sensors. The input is the user's online behavior data and emotional state data, and the output is the collected user dataset.

[0420] Step 4:

[0421] The server uses an emotion analysis engine to analyze the user's current emotional state. In this step, the emotion analysis engine classifies the user's psychological state based on the collected emotional state data. The input is the user's emotional state data, and the output is the analyzed emotional state.

[0422] Step 5:

[0423] The server uses a generative AI model to generate customized XR videos and recommendations based on the emotion analysis results and online behavior data. In this step, the generative AI creates content tailored to the user's specific needs and interests. The input is the emotion analysis results and online behavior data, and the output is customized XR videos and product recommendations.

[0424] Step 6:

[0425] The server uses 3D modeling tools such as Unity or Unreal Engine to specifically render the generated customized XR video. Here, the input is the generated customized XR video data, and the output is the actual viewable XR video.

[0426] Step 7:

[0427] The server sends the generated customized XR video and product recommendation information to the user's device. In this step, the server transfers the data via the notification system, and the user receives it. The input is the customized XR video and product recommendation information, and the output is the data sent to the user's device.

[0428] Step 8:

[0429] The device displays the customized XR video and product recommendation information received to the user. The user views the video in real time through the user's device display or smart glasses. The input is the customized data sent to the user's device, and the output is the visual display to the user.

[0430] This processing step allows users to receive optimized information and advertisements in real time based on their emotional and behavioral data.

[0431] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0432] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0433] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0434] [Second embodiment]

[0435] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0436] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0437] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0438] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0439] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0440] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0441] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0442] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0443] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0444] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0445] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0446] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0447] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI, and also provides customized advertisements based on the user's online behavior and preferences. The specific operation form and program processing of this system are explained below.

[0448] Providing real-time news

[0449] The server collects news data from multiple news sources (APIs, RSS feeds, news websites, etc.). The server then uses generative AI to analyze the collected data and extract key information. The server generates XR video based on the extracted information and sends the data to the user's device. The device visualizes the received XR video data, and the user watches the XR video.

[0450] Specific examples

[0451] 1. The server retrieves news data through the news API.

[0452] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[0453] 3. The server uses tools such as Unity or Unreal Engine to generate important information as XR images.

[0454] 4. The server sends the generated XR video data to the user's device, which then displays the data, allowing the user to watch the news in an immersive manner.

[0455] Providing virtual sports experiences

[0456] The server collects live sports data (APIs, sports websites, real-time data feeds, etc.). The server then uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. The server generates a virtual sports game video based on the extracted data and sends the data to the user's device. The device visualizes the received virtual sports video, allowing the user to watch the game.

[0457] Specific examples

[0458] 1. The server retrieves live match data through the sports API.

[0459] 2. The server analyzes the match using a generated AI and extracts important highlights.

[0460] 3. The server uses 3D modeling tools to virtually generate footage of the match.

[0461] 4. The server sends the generated virtual sports video data to the user's device, which then displays this data, allowing the user to experience an immersive game.

[0462] Providing customized advertising

[0463] The server collects data about a user's online behavior and preferences from cookies and web traffic. The server then uses generative AI to analyze the collected data and generate ads optimized for individual users. The server then sends the customized ads it generates to the user's device, which displays them. The user views the ads and, if interested, accesses a page for additional information or a purchase.

[0464] Specific examples

[0465] 1. The server uses web analysis tools to collect user behavior data.

[0466] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[0467] 3. The server then generates a personalized ad based on this information.

[0468] 4. The server sends the generated advertising data to the user's device, which displays the advertisement, allowing the user to access the advertisement and, if interested, go directly to a page for more information or to purchase.

[0469] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements, thereby realizing fast and accurate information provision and improved advertising effectiveness in an aging society or during disasters.

[0470] The processing flow will be explained below.

[0471] Providing real-time news

[0472] Step 1:

[0473] The server collects the latest news data from multiple news sources (e.g., news APIs, RSS feeds, news websites).

[0474] Step 2:

[0475] The server uses generated AI to analyze the collected news data and extract important keywords and events, specifically using natural language processing (NLP) algorithms.

[0476] Step 3:

[0477] Based on the key information extracted by the server, XR images (e.g., 3D models and infographics) are generated using engines such as Unity or Unreal Engine.

[0478] Step 4:

[0479] The server sends the generated XR video data to the user's device.

[0480] Step 5:

[0481] The device displays the received XR video data, allowing users to watch and experience the news in an immersive format.

[0482] Providing virtual sports experiences

[0483] Step 1:

[0484] A server collects live sports data (e.g., sports API, real-time data feeds).

[0485] Step 2:

[0486] The server uses generative AI to analyze the collected sports data and extract important highlights and key points from the match, such as scores, player movements, and interesting scenes.

[0487] Step 3:

[0488] The server generates a virtually recreated sports game video based on the extracted data, again using engines such as Unity and Unreal Engine.

[0489] Step 4:

[0490] The server transmits the generated virtual sports video data to the user's terminal.

[0491] Step 5:

[0492] The device displays the received virtual sports video data, allowing users to watch it and experience it as if they were watching a real game.

[0493] Providing customized advertising

[0494] Step 1:

[0495] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[0496] Step 2:

[0497] The server uses generated AI to analyze the collected data and identify individual users' interests and behavioral patterns, specifically by analyzing their browsing history and click patterns.

[0498] Step 3:

[0499] Based on the analysis results, the server generates individually optimized advertisements, which are customized to the user's interests.

[0500] Step 4:

[0501] The server transmits the generated customized advertisement to the user's terminal.

[0502] Step 5:

[0503] The device displays the received advertisement, and if the user is interested, they can use a QR code or clickable element to access more information or a purchase page.

[0504] Through these processing steps, users can efficiently receive important news, virtual sports experiences, and customized advertising in real time.

[0505] Example 1

[0506] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0507] There is a growing need for fast and accurate information provision in an aging society and during disasters. There is also a demand for customized advertising that meets individual user needs. Conventional methods are difficult to effectively address these challenges, so a system that can provide highly accurate information in real time and a customized experience is needed.

[0508] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0509] In this invention, the server includes means for collecting data in real time from multiple information sources using a generation AI and analyzing and extracting important information, means for generating mixed reality video based on the extracted information, and means for transmitting the generated mixed reality video to a user terminal and allowing the user to view it. This allows important information to be provided quickly in real time in an immersive format, making it possible to provide information in an aging society or during disasters.

[0510] "Generative AI" is an artificial intelligence technology that uses techniques such as machine learning and deep learning to generate and analyze data and extract specific information.

[0511] "Source" refers to the original media or platform that provides the data, such as an API, RSS feed, news website, or sports data feed.

[0512] "Real-time" refers to a situation in which data is generated, acquired, and analyzed almost simultaneously with real time, with extremely little delay.

[0513] "Data collection" is the process by which the server obtains data from external sources and stores it in its database.

[0514] "Analysis" is the process of processing collected data using generative AI to extract important information.

[0515] "Important information" refers to keywords, events, or phenomena that are particularly noteworthy among the collected data.

[0516] "Mixed reality video" refers to video content that combines the real world and the virtual world, generated using virtual reality (VR) or augmented reality (AR) technology.

[0517] "User device" refers to electronic devices used by users, such as computers, smartphones, tablets, and VR headsets.

[0518] "To allow a user to view" refers to the act of making a video or content available for a user to view through a user terminal.

[0519] "Live event data" refers to data that includes information about an event that is unfolding in real time, such as a sports game or a concert.

[0520] A "highlight" is a particularly important or noticeable part of an event or piece of data.

[0521] "Virtually recreated event footage" refers to footage that is virtually generated based on an actual event using 3D modeling and animation.

[0522] "Online behavior" refers to all the activities a user engages in on the Internet, including browsing, clicking, and purchasing history.

[0523] "Customized advertising" refers to advertising generated based on a user's individual interests and behavioral patterns.

[0524] This invention is a system that uses generative AI to collect data from multiple sources in real time, extract and analyze important information, and provide it to users. The system of the present invention provides visual mixed reality images to the user terminal.

[0525] Specific system configuration

[0526] Data collection

[0527] A server collects news and live event data in real time from multiple sources (APIs, RSS feeds, news websites, etc.) In this process, the server periodically sends HTTP requests to API endpoints and receives data in JSON format.

[0528] Data analysis and key information extraction

[0529] The server analyzes the collected data using a generative AI model. This analysis automatically extracts important keywords and events. For example, by providing the generative AI model with a prompt such as "Analyze the news data and extract important information," the server obtains the AI's analysis results.

[0530] Mixed reality image generation

[0531] The server uses the extracted information to generate mixed reality images using 3D modeling tools such as Unity or Unreal Engine. For example, a prompt such as "simulate a news event" can be sent to Unity, and a 3D scene can be constructed based on the simulation results.

[0532] Video data transmission

[0533] The server sends the generated mixed reality video data to the user's device using a network protocol (e.g., WebSocket or HTTP / 2) to stream the video data with low latency.

[0534] Visualization of video data

[0535] The device visualizes the received mixed reality video data and allows the user to view it. A dedicated application runs on the device, decodes the received data, and displays it on a VR headset or AR display.

[0536] Specific examples

[0537] News data provision

[0538] 1. The server retrieves news data through the news API from a URL such as "https: / / newsapi.org / v2 / top-headlines?country=jp&apiKey=YOUR_API_KEY".

[0539] 2. The server uses the generated AI to analyze the data with the prompt, "Analyze the news data and extract important information."

[0540] 3. The server uses Unity to generate a 3D scene based on a "simulated news event."

[0541] 4. The server sends the generated XR video data to the user's device.

[0542] 5. The device visualizes the received data, and the user watches an immersive news video on a VR headset.

[0543] Providing virtual sports experiences

[0544] 1. The server retrieves live match data through the sports API.

[0545] 2. The server uses a generative AI to analyze the data with the prompt, "Extract key highlights from the match data."

[0546] 3. The server uses a 3D modeling tool to virtually generate a "reenactment of the goal scene."

[0547] 4. The server sends the generated virtual sports video data to the user's device.

[0548] 5. The device visualizes the received data, allowing the user to watch an immersive game video.

[0549] Providing customized advertising

[0550] 1. The server uses web analysis tools to collect user behavior data.

[0551] 2. The server uses the generation AI to analyze the data with the prompt, "Analyze user behavior data and generate ads based on it."

[0552] 3. The server uses generated AI to generate individually customized ads.

[0553] 4. The server sends the generated advertisement to the user's device.

[0554] 5. The device displays the received advertisement, and if the user is interested, they can access the page for more information or to purchase.

[0555] In this way, the present invention provides a system that allows for fast and accurate information delivery and a customized experience.

[0556] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0557] Providing real-time news

[0558] Processing Steps

[0559] Step 1:

[0560] The server collects news data from multiple sources (APIs, RSS feeds, news websites, etc.). Specifically, the server periodically sends HTTP requests to API endpoints to retrieve data in JSON format. (Input) News API endpoint URL, (Output) JSON-formatted news data.

[0561] Step 2:

[0562] The server uses a generative AI model to analyze the collected news data. It uses the collected JSON data as input and provides the prompt, "Analyze the news data and extract important information," to the generative AI model, which then extracts important keywords and event information. (Input) JSON-formatted news data, prompt, (Output) Analysis results including important keywords and event information.

[0563] Step 3:

[0564] The server uses 3D modeling tools such as Unity or Unreal Engine to generate XR video based on the analysis results. The 3D modeling tool is given a prompt statement, "simulation of a news event," and a 3D scene is constructed based on key information. (Input) Analysis results, prompt statement, (Output) Generated XR video data.

[0565] Step 4:

[0566] The server sends the generated XR video data to the user's device. The server uses network protocols such as WebSocket or HTTP / 2 to stream the data with low latency. (Input) Generated XR video data. (Output) Transmission to the user's device is complete.

[0567] Step 5:

[0568] The device visualizes the received XR video data and the user views it. A dedicated application on the device decodes the data and displays the immersive video on a VR headset or AR display. (Input) Received XR video data, (Output) Visualized video (viewed by the user).

[0569] Providing virtual sports experiences

[0570] Processing Steps

[0571] Step 1:

[0572] The server uses the Sports API to collect live match data. The server sends a request to the API endpoint to retrieve data such as match scores and player movements in JSON format. (Input) Sports API endpoint URL, (Output) Live match data in JSON format.

[0573] Step 2:

[0574] The server uses a generative AI model to analyze live match data and extract important highlights. The generative AI model is given a prompt statement, "Please extract key highlights from the match data." The input is the live match data in JSON format, the prompt statement, and the output is the analysis results including important highlight information.

[0575] Step 3:

[0576] The server uses a 3D modeling tool to generate virtual sports footage based on the extracted highlight information. The 3D modeling tool is given the prompt "Reproduce the goal scene" to recreate the game scene. (Input) Highlight information, prompt, (Output) Generated virtual sports footage data.

[0577] Step 4:

[0578] The server sends the generated virtual sports video data to the user device. The server uses WebSocket to stream the data to the user device with low latency. (Input) Generated virtual sports video data, (Output) Transmission to the user device completed.

[0579] Step 5:

[0580] The device visualizes the received virtual sports video data and the user watches it. A dedicated application decodes the data, allowing the user to watch the game immersively using a VR headset. (Input) Received virtual sports video data, (Output) Visualized video (user viewing).

[0581] Providing customized advertising

[0582] Processing Steps

[0583] Step 1:

[0584] The server uses web analysis tools to collect data on users' online behavior. It collects cookies and web traffic data and logs users' browsing history and click behavior. (Input) User access logs and cookies, (Output) User online behavior data.

[0585] Step 2:

[0586] The server uses a generative AI model to analyze the behavioral data and determine the user's interests and behavioral patterns. It then provides the generative AI model with a prompt: "Analyze the user's behavioral data and generate advertisements based on it." (Input) The user's online behavioral data, the prompt, and (Output) the analysis results, including appropriate customized advertisement information.

[0587] Step 3:

[0588] The server uses the generative AI model to generate individually customized ads. It generates ads based on the analysis results and designs advertising campaigns. (Input) Analysis results, prompt text, (Output) Generated customized ads.

[0589] Step 4:

[0590] The server sends the generated customized advertisement to the user terminal. The server sends the advertisement data to the user terminal using the HTTP protocol. (Input) Generated customized advertisement, (Output) Transmission to the user terminal completed.

[0591] Step 5:

[0592] The device displays the received customized advertisement and the user views it. The advertisement is displayed in a browser or an application dedicated to displaying advertisements, allowing the user to access advertisements that interest them. (Input) Received customized advertisement, (Output) Displayed advertisement (viewed by the user).

[0593] (Application example 1)

[0594] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0595] There is a need for systems that can provide fast and accurate information in an aging society or during disasters. There is also a need for effective means of providing information so that security guards can patrol efficiently. While satisfying these requirements, there is also a need for systems that can display customized advertisements according to user preferences.

[0596] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0597] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing security-related information using the generation AI and customizing and providing optimal patrol routes and warning information, and means for displaying the customized information on the smart glasses, thereby enabling prompt and accurate information provision and efficient security management.

[0598] "Generative AI" is a system that uses artificial intelligence techniques to generate, analyze, and learn from data.

[0599] "News source" refers to an API, RSS feed, news website, or other source that provides news data.

[0600] "News Data" refers to articles, reports, footage, and other news-related information collected from news sources.

[0601] "Important information" refers to information such as key points, important keywords, and events extracted from news data.

[0602] "XR video" refers to video generated using technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0603] "User terminal" refers to a device used by a user, such as a smartphone, tablet, smart glasses, or head-mounted display.

[0604] "Security-related information" refers to patrol routes, warnings, alarm information, etc. required by security guards.

[0605] A "patrol route" refers to the optimal route for a security guard to patrol.

[0606] "Warning information" refers to data that provides information on where particular caution should be exercised in a particular situation or location.

[0607] "Smart glasses" refers to a glasses-type device equipped with a display for displaying information.

[0608] "Customized advertising" refers to advertising content that is optimized based on a user's online behavior and preferences.

[0609] In this invention, the following system configuration is adopted to realize an application for smart glasses specialized for security services.

[0610] First, the server collects news data in real time from news sources, such as news APIs, RSS feeds, and news websites. Then, the server uses generative AI to analyze the collected news data and extract important information. A possible generative AI model for this purpose is OpenAI GPT-4.

[0611] Based on the extracted important information, the server generates XR images. It is recommended to use a 3D modeling tool such as Unity or Unreal Engine. The generated XR images are then sent to the user's device, such as smart glasses.

[0612] The server also analyzes security-related information in real time. It uses AI to analyze patrol routes and warning information and provides customized information. This information is also displayed on the smart glasses. This allows security guards to quickly obtain information on optimal patrol routes and important points to pay attention to.

[0613] In addition, the server collects and analyzes users' online behavior data and past patrol data. It uses generative AI to analyze this data and generate advertising information optimized for each individual user. The generated advertising information is then sent to the user's device and displayed on the smart glasses.

[0614] Specific examples

[0615] Security guard A wears smart glasses while on duty. The glasses display the latest earthquake information and evacuation instructions in real time as XR video. They also suggest optimal patrol routes based on past patrol data analyzed by the server. Additionally, the glasses display advertisements customized based on A's web browsing history.

[0616] Prompt Sentence Examples

[0617] Extract important information from the following news data: {News Data}

[0618] This invention allows security guards to take action quickly and accurately, realizing efficient security management. In addition, by displaying advertisements according to user preferences, it is expected that the effectiveness of advertisements will be improved.

[0619] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0620] Step 1:

[0621] The server collects news data from news sources, specifically, news APIs, RSS feeds, news websites, etc., via API requests. The input is the URL or API endpoint of the news source, and the output is the data of the news article or report.

[0622] Step 2:

[0623] The server analyzes the collected news data using generative AI to extract important information. The input is the news data collected in the previous step, and important keywords and events are extracted by sending prompts to the generative AI model. The output is the extracted important information, which is used in subsequent steps.

[0624] Step 3:

[0625] The server generates XR video based on the extracted key information. For this purpose, it uses 3D modeling tools such as Unity or Unreal Engine. The input is the key information extracted in step 2, and the output is the generated XR video data.

[0626] Step 4:

[0627] The server sends the generated XR video data to the user device. The input is the XR video data generated in step 3, and also includes the address of the destination user device. The output is a message that the data was successfully sent to the user device.

[0628] Step 5:

[0629] The user device visualizes the XR video data received from the server and displays it to the user. The input is the XR video data sent from the server, and the output is the video displayed on a display such as smart glasses.

[0630] Step 6:

[0631] The server collects security-related information and analyzes it using generative AI. The input is security data (e.g., security logs, surveillance camera feeds), and the analysis extracts optimal patrol routes and warning information. The output is customized security information.

[0632] Step 7:

[0633] The server sends customized patrol routes and warning information to the smart glasses. The input is the security information generated in step 6, and the output is a data transmission success message to the smart glasses.

[0634] Step 8:

[0635] The smart glasses visualize customized patrol routes and warning information received from the server and display it to the user. The input is the security information sent from the server, and the output is the information displayed on the smart glasses' display.

[0636] Step 9:

[0637] The server collects the user's online behavior data and analyzes it using Generative AI. The input is the user's online behavior data (e.g., web traffic, cookies) and analyzes the user's interests and behavioral patterns. The output is advertising information optimized for each individual user.

[0638] Step 10:

[0639] The server sends the generated customized advertisement to the user terminal, where the input is the advertisement information generated in step 9, and the output is a data transmission success message to the user terminal.

[0640] Step 11:

[0641] The user terminal displays the customized advertisement received from the server. The input is the advertisement data sent from the server, and the output is the advertisement video that the user can view.

[0642] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0643] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI and an emotion engine, and also provides customized advertisements based on the user's online behavior and emotions. The specific operation form and program processing of this system are explained below.

[0644] Providing real-time news

[0645] The server collects news data from multiple news sources (e.g., news APIs, RSS feeds, and news websites). The server then uses generative AI to analyze the collected news data and extract important keywords and events. The server then uses an emotion engine to recognize the user's emotions and customizes the XR video of the news data based on the user's emotions. This ensures that when the video is sent to the user's device, it contains content that corresponds to the user's specific emotions. The device displays this customized XR video data, allowing the user to watch the news.

[0646] Specific examples

[0647] 1. The server retrieves news data through the news API.

[0648] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[0649] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the news content.

[0650] 4. The server generates customized XR footage using tools such as Unity or Unreal Engine.

[0651] 5. The server sends the customized XR video data generated to the user's device, which then displays it, allowing the user to watch news content tailored to their emotions in an immersive format.

[0652] Providing virtual sports experiences

[0653] The server collects live sports data (e.g., sports APIs, real-time data feeds). Then, the server uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. Furthermore, the server uses an emotion engine to recognize the user's emotions and customize the virtual sports game video based on the user's emotions. The device displays this customized virtual sports video for the user to watch.

[0654] Specific examples

[0655] 1. The server retrieves live match data through the sports API.

[0656] 2. The server analyzes the match using a generated AI and extracts important highlights.

[0657] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the content of the game footage.

[0658] 4. The server uses 3D modeling tools to generate a customized virtual match.

[0659] 5. The server sends the generated customized virtual sports video data to the user's device, which then displays it, allowing the user to experience a match that matches their emotions in an immersive format.

[0660] Providing customized advertising

[0661] The server collects users' online behavior data from cookies and web traffic. The server then uses generative AI to analyze the collected data and identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions and further customize the advertising content. The server then sends the customized advertisements it generates to the user's device, which displays them. The user views the advertisements and, if interested, accesses a page for additional information or to purchase.

[0662] Specific examples

[0663] 1. The server uses web analysis tools to collect user behavior data.

[0664] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[0665] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the advertising content.

[0666] 4. The server then generates a personalized ad based on this information.

[0667] 5. The server sends the generated advertising data to the user's device, which displays it, allowing the user to access an advertisement with content that corresponds to their emotions, and if they are interested, they will be taken directly to a page with more information or to a purchase page.

[0668] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements based on their emotions, which will enable the provision of fast and accurate information in an aging society or during disasters, as well as further improving advertising effectiveness.

[0669] The processing flow will be explained below.

[0670] Providing real-time news

[0671] Step 1:

[0672] The server collects news data in real time from multiple news sources such as news APIs, RSS feeds, and news websites.

[0673] Step 2:

[0674] The news data collected by the server is analyzed using generative AI to identify important keywords and events.

[0675] Step 3:

[0676] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[0677] Step 4:

[0678] The server customizes news content based on the user's emotional data, adjusting the tone and content to match specific emotions.

[0679] Step 5:

[0680] The server generates customized news data as XR video using tools such as Unity and Unreal Engine.

[0681] Step 6:

[0682] The server sends the generated customized XR video data to the user's device.

[0683] Step 7:

[0684] The device displays the received customized XR video data, and the user watches it to experience customized news tailored to their emotions in an immersive format.

[0685] Providing virtual sports experiences

[0686] Step 1:

[0687] The server collects live sports data from sports APIs, real-time data feeds, etc.

[0688] Step 2:

[0689] The server uses generated AI to analyze the collected sports data and extract important highlights and key points from the game.

[0690] Step 3:

[0691] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[0692] Step 4:

[0693] The server customizes the game footage based on the user's emotional data, adjusting visual effects and narration to match specific emotions.

[0694] Step 5:

[0695] The server generates customized virtual sports game footage using 3D modeling tools.

[0696] Step 6:

[0697] The server transmits the generated customized virtual sports video data to the user's terminal.

[0698] Step 7:

[0699] The terminal displays the received customized virtual sports video data, and the user watches the video data to experience a customized match according to their emotions in an immersive format.

[0700] Providing customized advertising

[0701] Step 1:

[0702] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[0703] Step 2:

[0704] The server uses generated AI to analyze the collected behavioral data and identify the user's interests and behavioral patterns.

[0705] Step 3:

[0706] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[0707] Step 4:

[0708] The server further customizes the optimized advertising content based on the user's emotional data, generating advertising content that corresponds to a specific emotion.

[0709] Step 5:

[0710] The server generates customized advertising data, with specific designs and messages tailored to the emotion.

[0711] Step 6:

[0712] The server transmits the generated customized advertisement data to the user's terminal.

[0713] Step 7:

[0714] The device displays the personalized ad it receives, the user watches the ad, engages with the ad with emotional content, and if interested, is taken directly to a page with more information or a purchase using a QR code or clickable element.

[0715] These processing steps effectively allow users to receive important news, virtual sports experiences, and customized advertising in real time, while the incorporation of an emotion engine provides a more personalized experience.

[0716] Example 2

[0717] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0718] In modern society, information overload, misinformation, biased reporting, and other factors make it difficult for users to quickly and accurately obtain the information they need. Furthermore, in an aging society or during disasters, providing information to individual users becomes increasingly important, but general media services have difficulty providing appropriate information based on the user's emotions and state. Furthermore, current advertising systems are one-sided and are unable to provide effective advertising that is in line with the user's emotions and interests.

[0719] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0720] In this invention, the server includes means for collecting and analyzing data from multiple information sources in real time using a generation AI and extracting important information, means for recognizing a user's emotions using an emotion engine, and means for generating XR video based on the extracted information and the recognized emotions. This enables users to receive appropriate information according to their emotions quickly and in an immersive format, which is expected to improve information provision in an aging society or during disasters, as well as further increase the effectiveness of advertising.

[0721] "Generative AI" refers to algorithms and models that use artificial intelligence technology to generate data such as text and images.

[0722] "Source" refers to the means by which information is obtained from digital sources, such as news APIs, RSS feeds, websites, etc.

[0723] "Real-time" means that data is processed and transmitted the instant it is generated or acquired.

[0724] "Emotion engine" refers to algorithms and models for analyzing and recognizing user emotions.

[0725] "XR video" refers to video formats including augmented reality (AR), virtual reality (VR), and mixed reality (MR).

[0726] "User terminal" refers to digital devices such as smartphones, tablets, and personal computers.

[0727] "Live Data" refers to dynamic data that is generated and captured in real time.

[0728] "Online behavioral data" refers to data about the history and patterns of a user's activities on the Internet.

[0729] "Customized advertising" refers to advertising that is optimized based on the interests and emotions of individual users.

[0730] This invention is a comprehensive media system that uses generative AI and an emotion engine to provide users with customized news, virtual sports experiences, and advertising. Specific embodiments of this system are described below.

[0731] News data provision

[0732] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to obtain news data in real time. The collected data undergoes text analysis using generative AI (e.g., GPT-4) to extract important keywords and events. The server then uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Based on the user's emotions, the server adjusts and customizes the content of the news data. The server then generates customized XR video using tools such as Unity or Unreal Engine. The generated XR video data is sent to the user's device, where the user can view it.

[0733] As a specific example, we will adopt a method of collecting the latest earthquake news and displaying information that will reassure users who are feeling anxious. An example of a prompt sentence would be, "Collect the latest earthquake news and display it in a format that will reassure users who are feeling anxious."

[0734] Providing virtual sports experiences

[0735] The server accesses sports APIs and real-time data feeds to obtain live sports data. The collected data is analyzed by generative AI to extract important highlights and key points from the match. The server then uses an emotion engine to recognize the user's emotions in real time. Based on this, the content of the match video is adjusted and customized. Specifically, it provides more thrilling footage to excited users. The server uses 3D modeling tools such as Blender and Maya to generate customized virtual match video. The generated video data is sent to the user's device, where the user can watch it.

[0736] As a concrete example, live soccer data is analyzed and videos that highlight goal scenes, fine plays, etc. An example of a prompt sentence is "Analyze live soccer data and generate exciting videos for excited users."

[0737] Providing customized advertising

[0738] The server collects online behavior data from cookies and web traffic. The collected data is analyzed by Generative AI to identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions in real time. Based on this, advertising content is adjusted and customized. Specifically, an excited user will be served an energetic advertisement. The server then uses Generative AI to generate a customized advertisement and sends this data to the user's device. The user views the advertisement on their device and, if interested, accesses a page for additional information or to purchase.

[0739] As a concrete example, a user's online behavior data is analyzed, and if the user is interested in a particular product, an advertisement mainly consisting of positive reviews and videos about that product is provided. An example of a prompt sentence would be, "Analyze the user's online behavior data and generate a positive advertisement for the product they are interested in."

[0740] This system allows users to receive emotionally appropriate information quickly and in an immersive format, which is expected to improve information provision in aging societies and during disasters, and further increase the effectiveness of advertising.

[0741] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0742] News data provision

[0743] Step 1: Gathering news data

[0744] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to collect news data in real time. Specifically, it obtains data in JSON format through a REST API. The input of this step is news data obtained from the sources, and the output is the collected, unparsed news data.

[0745] Step 2: Analyzing news data and extracting keywords

[0746] The server uses a generative AI (e.g., GPT-4) to analyze the collected news data. It uses a text analysis algorithm to extract important keywords and events. The input of this step is the collected news data, and the output is the extracted important keywords and event information.

[0747] Step 3: Recognizing user emotions

[0748] The server uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Specifically, it analyzes the user's past behavioral data and real-time sensor data to estimate the user's emotional state. The input for this step is the user's real-time and historical data, and the output is the user's emotional state.

[0749] Step 4: Customize your news content

[0750] The server adjusts and customizes the content of the news data based on the emotion recognition results. For example, it restructures the content by emphasizing information that gives a sense of security to a user who is feeling anxious. The inputs for this step are extracted keywords, event information, and the user's emotional state, and the output is customized news content.

[0751] Step 5: Generate XR footage

[0752] The server uses tools such as Unity or Unreal Engine to generate customized XR footage, specifically creating a 3D reproduction of the news and related visual effects. The input for this step is the customized news content, and the output is the generated XR footage.

[0753] Step 6: Send and view your customized XR footage

[0754] The server sends the generated customized XR video data to the user's device. The device displays the received video data, and the user watches it. The input to this step is the generated XR video data, and the output is the video displayed on the user's device.

[0755] Providing virtual sports experiences

[0756] Step 1: Collecting live sports data

[0757] The server accesses sports APIs and real-time data feeds to collect live sports data. Specifically, it retrieves data in JSON format through REST APIs. The input of this step is the sports data retrieved from the source, and the output is the collected, unparsed sports data.

[0758] Step 2: Analyzing sports data and extracting highlights

[0759] The server uses the generative AI to analyze the collected sports data. It uses text analysis and video analysis algorithms to extract important highlights and critical moments. The input of this step is the collected sports data, and the output is the extracted highlight information.

[0760] Step 3: Recognizing user emotions

[0761] The server uses an emotion engine to recognize the user's emotions in real time, using the user's past viewing history and real-time feedback data. The input of this step is the user's real-time and historical data, and the output is the user's emotional state.

[0762] Step 4: Customize your video content

[0763] The server adjusts and customizes the content of the game footage based on the emotion recognition results. For example, it edits the footage to be more thrilling for an excited user. The input of this step is the extracted highlight information and the user's emotional state, and the output is customized game footage.

[0764] Step 5: Generate virtual matches

[0765] The server generates customized virtual game footage using 3D modeling tools such as Blender or Maya. The input of this step is the customized game content, and the output is the generated virtual game footage data.

[0766] Step 6: Send and view your customized virtual game footage

[0767] The server sends the generated virtual game video data to the user terminal. The terminal displays the received video data, and the user watches it. The input to this step is the generated virtual game video data, and the output is the video displayed on the user terminal.

[0768] Providing customized advertising

[0769] Step 1: Collect online behavior data

[0770] The server collects online behavior data from cookies and web traffic. Specifically, it obtains the data using web analytics tools such as Google Analytics. The input of this step is the user's online behavior data, and the output is the collected raw data.

[0771] Step 2: Analyze interests and behavioral patterns

[0772] The server uses the generated AI to analyze the collected data. It uses an algorithm to identify individual user interests and behavioral patterns. The input of this step is the collected behavioral data, and the output is analyzed user interest and behavioral pattern information.

[0773] Step 3: Recognizing user emotions

[0774] The server uses an emotion engine to recognize the user's emotions in real time, using real-time data collected from the webcam and microphone. The input of this step is real-time data and historical data, and the output is the user's emotional state.

[0775] Step 4: Customize your ad content

[0776] The server adjusts and customizes the advertisement content based on the emotion recognition results. For example, if the user is excited, it will provide an energetic presentation. The input of this step is the analyzed interest information and emotional state, and the output is customized advertisement content.

[0777] Step 5: Generate a personalized ad

[0778] The server uses generative AI to generate individually optimized ads, adjusting visual effects and wording. The input for this step is the customized ad content, and the output is the generated ad data.

[0779] Step 6: Send and display personalized ads

[0780] The server sends the generated advertisement data to the user terminal. The terminal displays it, and the user watches the advertisement. The input of this step is the generated advertisement data, and the output is the advertisement displayed on the user terminal.

[0781] (Application example 2)

[0782] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0783] Conventional news, sports video, and advertising systems lack personalization based on individual users' emotions and online behavior data, limiting the effectiveness of information provision. In particular, physical stores lack the means to provide users with appropriate product recommendations and promotional information, creating a demand for real-time, emotion-based customized information delivery. A system is needed to solve these issues and improve user experience.

[0784] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0785] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing the user's emotional state, means for customizing the XR video based on the emotion analysis results, and means for providing personalized product recommendation information to the user in a store, thereby enabling the provision of customized information based on the user's emotions and online behavior data.

[0786] "Generative AI" is an artificial intelligence algorithm that generates and analyzes information from massive amounts of data.

[0787] "News source" refers to the online platform or information provider that provides news data.

[0788] "News data" refers to a collection of information reported as news, including text, images, and video.

[0789] "XR video" refers to augmented reality (AR), virtual reality (VR), and mixed reality (MR) video that combines the real world and the virtual world.

[0790] A "user terminal" is a device for receiving and displaying information, and includes smartphones, smart glasses, head-mounted displays, etc.

[0791] "Emotional state" refers to the user's current psychological feelings, such as "joy," "anger," or "sadness."

[0792] "Emotion analysis" refers to technology for recognizing and classifying a user's emotional state.

[0793] "Customization" refers to tailoring content and information to best suit individual users.

[0794] "Recommended information" refers to information about products and services that are suggested to users.

[0795] "Products" refers to goods and services sold in physical stores.

[0796] "In-store" refers to the physical location where goods and services are provided.

[0797] "Personalization" refers to the unique tailoring of information or services based on a user's individual attributes, behavior, or emotions.

[0798] "Promotional information" refers to information about sales, campaigns, events, etc. for sales promotion purposes.

[0799] This invention realizes a system that uses generative AI and an emotion engine to provide individually customized information and advertisements to users, which can improve the user's shopping experience, especially in physical stores.

[0800] System Overview:

[0801] This system uses generative AI to collect and analyze real-time news data, extract and generate important information, analyze the user's emotional state, and customize XR video and product recommendations based on the results. Finally, the customized information is sent to the user's device, allowing them to view it.

[0802] Hardware and software used:

[0803] Hardware: User devices such as smartphones, smart glasses, and head-mounted displays.

[0804] Software: Generative AI models, sentiment analysis engines, notification systems, 3D modeling tools such as Unity and Unreal Engine.

[0805] Processing Details:

[0806] 1. Data Collection:

[0807] The server collects news data in real time from multiple news sources, such as news APIs and RSS feeds, as well as users' online behavioral data and emotional states.

[0808] 2. Data Analysis:

[0809] Generative AI models are used to analyze collected news data and extract important keywords and events, as well as users' online behavior data, to identify the most relevant information for each individual user.

[0810] 3. Emotion analysis:

[0811] A sentiment analysis engine is used to analyze the user's current emotional state, which is then used to customize news and product recommendations.

[0812] 4. Information generation:

[0813] Based on the analysis results, a generative AI model is used to generate customized XR footage and recommendations, including 3D modeling using Unity or Unreal Engine.

[0814] 5. Information provision:

[0815] The generated customized information is sent to the user's device via the notification system, allowing the user to view news and product recommendations in real time according to their emotional state and interests.

[0816] Examples:

[0817] For example, consider a case where a user has a search history of "shoes" and "watches," has a history of purchasing "jeans," and is currently in a "happy" emotional state. Based on this data, the server uses an emotion analysis engine to analyze the user's emotions and generate recommendations such as "casual shoes" and "sports watches." At the same time, it notifies the user of current sales and campaign information.

[0818] Example prompt sentence:

[0819] "User ID 12345's current emotional state is 'happy', and his / her online behavior data includes 'shoes' and 'watches' in his / her search history and 'jeans' in his / her purchase history. Based on this, please generate recommended products and related promotion information for the user."

[0820] This system will enable users to receive information and advertisements in real time that are optimized to their emotions and behavior, enabling a more personalized shopping experience, especially in physical stores.

[0821] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0822] Step 1:

[0823] The server collects news data in real time from multiple news sources such as news APIs and RSS feeds. In this data collection process, it loads media data such as text, images, and videos from the news sources. The input is data from the news sources, and the output is a set of collected news data.

[0824] Step 2:

[0825] The server uses a generative AI model to analyze the news data collected and extract important keywords and events. Here, the input news data is analyzed using natural language processing technology to perform summarization and topic extraction. The output is the extracted important keywords and events.

[0826] Step 3:

[0827] The server collects the user's online behavior data and emotional state data. At this stage, it obtains the user's online behavior data, such as browsing history, search history, and purchase history, and also collects emotional state data from smart devices and sensors. The input is the user's online behavior data and emotional state data, and the output is the collected user dataset.

[0828] Step 4:

[0829] The server uses an emotion analysis engine to analyze the user's current emotional state. In this step, the emotion analysis engine classifies the user's psychological state based on the collected emotional state data. The input is the user's emotional state data, and the output is the analyzed emotional state.

[0830] Step 5:

[0831] The server uses a generative AI model to generate customized XR videos and recommendations based on the emotion analysis results and online behavior data. In this step, the generative AI creates content tailored to the user's specific needs and interests. The input is the emotion analysis results and online behavior data, and the output is customized XR videos and product recommendations.

[0832] Step 6:

[0833] The server uses 3D modeling tools such as Unity or Unreal Engine to specifically render the generated customized XR video. Here, the input is the generated customized XR video data, and the output is the actual viewable XR video.

[0834] Step 7:

[0835] The server sends the generated customized XR video and product recommendation information to the user's device. In this step, the server transfers the data via the notification system, and the user receives it. The input is the customized XR video and product recommendation information, and the output is the data sent to the user's device.

[0836] Step 8:

[0837] The device displays the customized XR video and product recommendation information received to the user. The user views the video in real time through the user's device display or smart glasses. The input is the customized data sent to the user's device, and the output is the visual display to the user.

[0838] This processing step allows users to receive optimized information and advertisements in real time based on their emotional and behavioral data.

[0839] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0840] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0841] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0842] [Third embodiment]

[0843] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0844] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0845] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0846] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0847] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0848] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0849] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0850] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0851] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0852] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0853] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0854] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0855] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI, and also provides customized advertisements based on the user's online behavior and preferences. The specific operation form and program processing of this system are explained below.

[0856] Providing real-time news

[0857] The server collects news data from multiple news sources (APIs, RSS feeds, news websites, etc.). The server then uses generative AI to analyze the collected data and extract key information. The server generates XR video based on the extracted information and sends the data to the user's device. The device visualizes the received XR video data, and the user watches the XR video.

[0858] Specific examples

[0859] 1. The server retrieves news data through the news API.

[0860] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[0861] 3. The server uses tools such as Unity or Unreal Engine to generate important information as XR images.

[0862] 4. The server sends the generated XR video data to the user's device, which then displays the data, allowing the user to watch the news in an immersive manner.

[0863] Providing virtual sports experiences

[0864] The server collects live sports data (APIs, sports websites, real-time data feeds, etc.). The server then uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. The server generates a virtual sports game video based on the extracted data and sends the data to the user's device. The device visualizes the received virtual sports video, allowing the user to watch the game.

[0865] Specific examples

[0866] 1. The server retrieves live match data through the sports API.

[0867] 2. The server analyzes the match using a generated AI and extracts important highlights.

[0868] 3. The server uses 3D modeling tools to virtually generate footage of the match.

[0869] 4. The server sends the generated virtual sports video data to the user's device, which then displays this data, allowing the user to experience an immersive game.

[0870] Providing customized advertising

[0871] The server collects data about a user's online behavior and preferences from cookies and web traffic. The server then uses generative AI to analyze the collected data and generate ads optimized for individual users. The server then sends the customized ads it generates to the user's device, which displays them. The user views the ads and, if interested, accesses a page for additional information or a purchase.

[0872] Specific examples

[0873] 1. The server uses web analysis tools to collect user behavior data.

[0874] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[0875] 3. The server then generates a personalized ad based on this information.

[0876] 4. The server sends the generated advertising data to the user's device, which displays the advertisement, allowing the user to access the advertisement and, if interested, go directly to a page for more information or to purchase.

[0877] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements, thereby realizing fast and accurate information provision and improved advertising effectiveness in an aging society or during disasters.

[0878] The processing flow will be explained below.

[0879] Providing real-time news

[0880] Step 1:

[0881] The server collects the latest news data from multiple news sources (e.g., news APIs, RSS feeds, news websites).

[0882] Step 2:

[0883] The server uses generated AI to analyze the collected news data and extract important keywords and events, specifically using natural language processing (NLP) algorithms.

[0884] Step 3:

[0885] Based on the key information extracted by the server, XR images (e.g., 3D models and infographics) are generated using engines such as Unity or Unreal Engine.

[0886] Step 4:

[0887] The server sends the generated XR video data to the user's device.

[0888] Step 5:

[0889] The device displays the received XR video data, allowing users to watch and experience the news in an immersive format.

[0890] Providing virtual sports experiences

[0891] Step 1:

[0892] A server collects live sports data (e.g., sports API, real-time data feeds).

[0893] Step 2:

[0894] The server uses generative AI to analyze the collected sports data and extract important highlights and key points from the match, such as scores, player movements, and interesting scenes.

[0895] Step 3:

[0896] The server generates a virtually recreated sports game video based on the extracted data, again using engines such as Unity and Unreal Engine.

[0897] Step 4:

[0898] The server transmits the generated virtual sports video data to the user's terminal.

[0899] Step 5:

[0900] The device displays the received virtual sports video data, allowing users to watch it and experience it as if they were watching a real game.

[0901] Providing customized advertising

[0902] Step 1:

[0903] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[0904] Step 2:

[0905] The server uses generated AI to analyze the collected data and identify individual users' interests and behavioral patterns, specifically by analyzing their browsing history and click patterns.

[0906] Step 3:

[0907] Based on the analysis results, the server generates individually optimized advertisements, which are customized to the user's interests.

[0908] Step 4:

[0909] The server transmits the generated customized advertisement to the user's terminal.

[0910] Step 5:

[0911] The device displays the received advertisement, and if the user is interested, they can use a QR code or clickable element to access more information or a purchase page.

[0912] Through these processing steps, users can efficiently receive important news, virtual sports experiences, and customized advertising in real time.

[0913] Example 1

[0914] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0915] There is a growing need for fast and accurate information provision in an aging society and during disasters. There is also a demand for customized advertising that meets individual user needs. Conventional methods are difficult to effectively address these challenges, so a system that can provide highly accurate information in real time and a customized experience is needed.

[0916] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0917] In this invention, the server includes means for collecting data in real time from multiple information sources using a generation AI and analyzing and extracting important information, means for generating mixed reality video based on the extracted information, and means for transmitting the generated mixed reality video to a user terminal and allowing the user to view it. This allows important information to be provided quickly in real time in an immersive format, making it possible to provide information in an aging society or during disasters.

[0918] "Generative AI" is an artificial intelligence technology that uses techniques such as machine learning and deep learning to generate and analyze data and extract specific information.

[0919] "Source" refers to the original media or platform that provides the data, such as an API, RSS feed, news website, or sports data feed.

[0920] "Real-time" refers to a situation in which data is generated, acquired, and analyzed almost simultaneously with real time, with extremely little delay.

[0921] "Data collection" is the process by which the server obtains data from external sources and stores it in its database.

[0922] "Analysis" is the process of processing collected data using generative AI to extract important information.

[0923] "Important information" refers to keywords, events, or phenomena that are particularly noteworthy among the collected data.

[0924] "Mixed reality video" refers to video content that combines the real world and the virtual world, generated using virtual reality (VR) or augmented reality (AR) technology.

[0925] "User device" refers to electronic devices used by users, such as computers, smartphones, tablets, and VR headsets.

[0926] "To allow a user to view" refers to the act of making a video or content available for a user to view through a user terminal.

[0927] "Live event data" refers to data that includes information about an event that is unfolding in real time, such as a sports game or a concert.

[0928] A "highlight" is a particularly important or noticeable part of an event or piece of data.

[0929] "Virtually recreated event footage" refers to footage that is virtually generated based on an actual event using 3D modeling and animation.

[0930] "Online behavior" refers to all the activities a user engages in on the Internet, including browsing, clicking, and purchasing history.

[0931] "Customized advertising" refers to advertising generated based on a user's individual interests and behavioral patterns.

[0932] This invention is a system that uses generative AI to collect data from multiple sources in real time, extract and analyze important information, and provide it to users. The system of the present invention provides visual mixed reality images to the user terminal.

[0933] Specific system configuration

[0934] Data collection

[0935] A server collects news and live event data in real time from multiple sources (APIs, RSS feeds, news websites, etc.) In this process, the server periodically sends HTTP requests to API endpoints and receives data in JSON format.

[0936] Data analysis and key information extraction

[0937] The server analyzes the collected data using a generative AI model. This analysis automatically extracts important keywords and events. For example, by providing the generative AI model with a prompt such as "Analyze the news data and extract important information," the server obtains the AI's analysis results.

[0938] Mixed reality image generation

[0939] The server uses the extracted information to generate mixed reality images using 3D modeling tools such as Unity or Unreal Engine. For example, a prompt such as "simulate a news event" can be sent to Unity, and a 3D scene can be constructed based on the simulation results.

[0940] Video data transmission

[0941] The server sends the generated mixed reality video data to the user's device using a network protocol (e.g., WebSocket or HTTP / 2) to stream the video data with low latency.

[0942] Visualization of video data

[0943] The device visualizes the received mixed reality video data and allows the user to view it. A dedicated application runs on the device, decodes the received data, and displays it on a VR headset or AR display.

[0944] Specific examples

[0945] News data provision

[0946] 1. The server retrieves news data through the news API from a URL such as "https: / / newsapi.org / v2 / top-headlines?country=jp&apiKey=YOUR_API_KEY".

[0947] 2. The server uses the generated AI to analyze the data with the prompt, "Analyze the news data and extract important information."

[0948] 3. The server uses Unity to generate a 3D scene based on a "simulated news event."

[0949] 4. The server sends the generated XR video data to the user's device.

[0950] 5. The device visualizes the received data, and the user watches an immersive news video on a VR headset.

[0951] Providing virtual sports experiences

[0952] 1. The server retrieves live match data through the sports API.

[0953] 2. The server uses a generative AI to analyze the data with the prompt, "Extract key highlights from the match data."

[0954] 3. The server uses a 3D modeling tool to virtually generate a "reenactment of the goal scene."

[0955] 4. The server sends the generated virtual sports video data to the user's device.

[0956] 5. The device visualizes the received data, allowing the user to watch an immersive game video.

[0957] Providing customized advertising

[0958] 1. The server uses web analysis tools to collect user behavior data.

[0959] 2. The server uses the generation AI to analyze the data with the prompt, "Analyze user behavior data and generate ads based on it."

[0960] 3. The server uses generated AI to generate individually customized ads.

[0961] 4. The server sends the generated advertisement to the user's device.

[0962] 5. The device displays the received advertisement, and if the user is interested, they can access the page for more information or to purchase.

[0963] In this way, the present invention provides a system that allows for fast and accurate information delivery and a customized experience.

[0964] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0965] Providing real-time news

[0966] Processing Steps

[0967] Step 1:

[0968] The server collects news data from multiple sources (APIs, RSS feeds, news websites, etc.). Specifically, the server periodically sends HTTP requests to API endpoints to retrieve data in JSON format. (Input) News API endpoint URL, (Output) JSON-formatted news data.

[0969] Step 2:

[0970] The server uses a generative AI model to analyze the collected news data. It uses the collected JSON data as input and provides the prompt, "Analyze the news data and extract important information," to the generative AI model, which then extracts important keywords and event information. (Input) JSON-formatted news data, prompt, (Output) Analysis results including important keywords and event information.

[0971] Step 3:

[0972] The server uses 3D modeling tools such as Unity or Unreal Engine to generate XR video based on the analysis results. The 3D modeling tool is given a prompt statement, "simulation of a news event," and a 3D scene is constructed based on key information. (Input) Analysis results, prompt statement, (Output) Generated XR video data.

[0973] Step 4:

[0974] The server sends the generated XR video data to the user's device. The server uses network protocols such as WebSocket or HTTP / 2 to stream the data with low latency. (Input) Generated XR video data. (Output) Transmission to the user's device is complete.

[0975] Step 5:

[0976] The device visualizes the received XR video data and the user views it. A dedicated application on the device decodes the data and displays the immersive video on a VR headset or AR display. (Input) Received XR video data, (Output) Visualized video (viewed by the user).

[0977] Providing virtual sports experiences

[0978] Processing Steps

[0979] Step 1:

[0980] The server uses the Sports API to collect live match data. The server sends a request to the API endpoint to retrieve data such as match scores and player movements in JSON format. (Input) Sports API endpoint URL, (Output) Live match data in JSON format.

[0981] Step 2:

[0982] The server uses a generative AI model to analyze live match data and extract important highlights. The generative AI model is given a prompt statement, "Please extract key highlights from the match data." The input is the live match data in JSON format, the prompt statement, and the output is the analysis results including important highlight information.

[0983] Step 3:

[0984] The server uses a 3D modeling tool to generate virtual sports footage based on the extracted highlight information. The 3D modeling tool is given the prompt "Reproduce the goal scene" to recreate the game scene. (Input) Highlight information, prompt, (Output) Generated virtual sports footage data.

[0985] Step 4:

[0986] The server sends the generated virtual sports video data to the user device. The server uses WebSocket to stream the data to the user device with low latency. (Input) Generated virtual sports video data, (Output) Transmission to the user device completed.

[0987] Step 5:

[0988] The device visualizes the received virtual sports video data and the user watches it. A dedicated application decodes the data, allowing the user to watch the game immersively using a VR headset. (Input) Received virtual sports video data, (Output) Visualized video (user viewing).

[0989] Providing customized advertising

[0990] Processing Steps

[0991] Step 1:

[0992] The server uses web analysis tools to collect data on users' online behavior. It collects cookies and web traffic data and logs users' browsing history and click behavior. (Input) User access logs and cookies, (Output) User online behavior data.

[0993] Step 2:

[0994] The server uses a generative AI model to analyze the behavioral data and determine the user's interests and behavioral patterns. It then provides the generative AI model with a prompt: "Analyze the user's behavioral data and generate advertisements based on it." (Input) The user's online behavioral data, the prompt, and (Output) the analysis results, including appropriate customized advertisement information.

[0995] Step 3:

[0996] The server uses the generative AI model to generate individually customized ads. It generates ads based on the analysis results and designs advertising campaigns. (Input) Analysis results, prompt text, (Output) Generated customized ads.

[0997] Step 4:

[0998] The server sends the generated customized advertisement to the user terminal. The server sends the advertisement data to the user terminal using the HTTP protocol. (Input) Generated customized advertisement, (Output) Transmission to the user terminal completed.

[0999] Step 5:

[1000] The device displays the received customized advertisement and the user views it. The advertisement is displayed in a browser or an application dedicated to displaying advertisements, allowing the user to access advertisements that interest them. (Input) Received customized advertisement, (Output) Displayed advertisement (viewed by the user).

[1001] (Application example 1)

[1002] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1003] There is a need for systems that can provide fast and accurate information in an aging society or during disasters. There is also a need for effective means of providing information so that security guards can patrol efficiently. While satisfying these requirements, there is also a need for systems that can display customized advertisements according to user preferences.

[1004] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1005] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing security-related information using the generation AI and customizing and providing optimal patrol routes and warning information, and means for displaying the customized information on the smart glasses, thereby enabling prompt and accurate information provision and efficient security management.

[1006] "Generative AI" is a system that uses artificial intelligence techniques to generate, analyze, and learn from data.

[1007] "News source" refers to an API, RSS feed, news website, or other source that provides news data.

[1008] "News Data" refers to articles, reports, footage, and other news-related information collected from news sources.

[1009] "Important information" refers to information such as key points, important keywords, and events extracted from news data.

[1010] "XR video" refers to video generated using technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[1011] "User terminal" refers to a device used by a user, such as a smartphone, tablet, smart glasses, or head-mounted display.

[1012] "Security-related information" refers to patrol routes, warnings, alarm information, etc. required by security guards.

[1013] A "patrol route" refers to the optimal route for a security guard to patrol.

[1014] "Warning information" refers to data that provides information on where particular caution should be exercised in a particular situation or location.

[1015] "Smart glasses" refers to a glasses-type device equipped with a display for displaying information.

[1016] "Customized advertising" refers to advertising content that is optimized based on a user's online behavior and preferences.

[1017] In this invention, the following system configuration is adopted to realize an application for smart glasses specialized for security services.

[1018] First, the server collects news data in real time from news sources, such as news APIs, RSS feeds, and news websites. Then, the server uses generative AI to analyze the collected news data and extract important information. A possible generative AI model for this purpose is OpenAI GPT-4.

[1019] Based on the extracted important information, the server generates XR images. It is recommended to use a 3D modeling tool such as Unity or Unreal Engine. The generated XR images are then sent to the user's device, such as smart glasses.

[1020] The server also analyzes security-related information in real time. It uses AI to analyze patrol routes and warning information and provides customized information. This information is also displayed on the smart glasses. This allows security guards to quickly obtain information on optimal patrol routes and important points to pay attention to.

[1021] In addition, the server collects and analyzes users' online behavior data and past patrol data. It uses generative AI to analyze this data and generate advertising information optimized for each individual user. The generated advertising information is then sent to the user's device and displayed on the smart glasses.

[1022] Specific examples

[1023] Security guard A wears smart glasses while on duty. The glasses display the latest earthquake information and evacuation instructions in real time as XR video. They also suggest optimal patrol routes based on past patrol data analyzed by the server. Additionally, the glasses display advertisements customized based on A's web browsing history.

[1024] Prompt Sentence Examples

[1025] Extract important information from the following news data: {News Data}

[1026] This invention allows security guards to take action quickly and accurately, realizing efficient security management. In addition, by displaying advertisements according to user preferences, it is expected that the effectiveness of advertisements will be improved.

[1027] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1028] Step 1:

[1029] The server collects news data from news sources, specifically, news APIs, RSS feeds, news websites, etc., via API requests. The input is the URL or API endpoint of the news source, and the output is the data of the news article or report.

[1030] Step 2:

[1031] The server analyzes the collected news data using generative AI to extract important information. The input is the news data collected in the previous step, and important keywords and events are extracted by sending prompts to the generative AI model. The output is the extracted important information, which is used in subsequent steps.

[1032] Step 3:

[1033] The server generates XR video based on the extracted key information. For this purpose, it uses 3D modeling tools such as Unity or Unreal Engine. The input is the key information extracted in step 2, and the output is the generated XR video data.

[1034] Step 4:

[1035] The server sends the generated XR video data to the user device. The input is the XR video data generated in step 3, and also includes the address of the destination user device. The output is a message that the data was successfully sent to the user device.

[1036] Step 5:

[1037] The user device visualizes the XR video data received from the server and displays it to the user. The input is the XR video data sent from the server, and the output is the video displayed on a display such as smart glasses.

[1038] Step 6:

[1039] The server collects security-related information and analyzes it using generative AI. The input is security data (e.g., security logs, surveillance camera feeds), and the analysis extracts optimal patrol routes and warning information. The output is customized security information.

[1040] Step 7:

[1041] The server sends customized patrol routes and warning information to the smart glasses. The input is the security information generated in step 6, and the output is a data transmission success message to the smart glasses.

[1042] Step 8:

[1043] The smart glasses visualize customized patrol routes and warning information received from the server and display it to the user. The input is the security information sent from the server, and the output is the information displayed on the smart glasses' display.

[1044] Step 9:

[1045] The server collects the user's online behavior data and analyzes it using Generative AI. The input is the user's online behavior data (e.g., web traffic, cookies) and analyzes the user's interests and behavioral patterns. The output is advertising information optimized for each individual user.

[1046] Step 10:

[1047] The server sends the generated customized advertisement to the user terminal, where the input is the advertisement information generated in step 9, and the output is a data transmission success message to the user terminal.

[1048] Step 11:

[1049] The user terminal displays the customized advertisement received from the server. The input is the advertisement data sent from the server, and the output is the advertisement video that the user can view.

[1050] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1051] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI and an emotion engine, and also provides customized advertisements based on the user's online behavior and emotions. The specific operation form and program processing of this system are explained below.

[1052] Providing real-time news

[1053] The server collects news data from multiple news sources (e.g., news APIs, RSS feeds, and news websites). The server then uses generative AI to analyze the collected news data and extract important keywords and events. The server then uses an emotion engine to recognize the user's emotions and customizes the XR video of the news data based on the user's emotions. This ensures that when the video is sent to the user's device, it contains content that corresponds to the user's specific emotions. The device displays this customized XR video data, allowing the user to watch the news.

[1054] Specific examples

[1055] 1. The server retrieves news data through the news API.

[1056] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[1057] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the news content.

[1058] 4. The server generates customized XR footage using tools such as Unity or Unreal Engine.

[1059] 5. The server sends the customized XR video data generated to the user's device, which then displays it, allowing the user to watch news content tailored to their emotions in an immersive format.

[1060] Providing virtual sports experiences

[1061] The server collects live sports data (e.g., sports APIs, real-time data feeds). Then, the server uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. Furthermore, the server uses an emotion engine to recognize the user's emotions and customize the virtual sports game video based on the user's emotions. The device displays this customized virtual sports video for the user to watch.

[1062] Specific examples

[1063] 1. The server retrieves live match data through the sports API.

[1064] 2. The server analyzes the match using a generated AI and extracts important highlights.

[1065] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the content of the game footage.

[1066] 4. The server uses 3D modeling tools to generate a customized virtual match.

[1067] 5. The server sends the generated customized virtual sports video data to the user's device, which then displays it, allowing the user to experience a match that matches their emotions in an immersive format.

[1068] Providing customized advertising

[1069] The server collects users' online behavior data from cookies and web traffic. The server then uses generative AI to analyze the collected data and identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions and further customize the advertising content. The server then sends the customized advertisements it generates to the user's device, which displays them. The user views the advertisements and, if interested, accesses a page for additional information or to purchase.

[1070] Specific examples

[1071] 1. The server uses web analysis tools to collect user behavior data.

[1072] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[1073] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the advertising content.

[1074] 4. The server then generates a personalized ad based on this information.

[1075] 5. The server sends the generated advertising data to the user's device, which displays it, allowing the user to access an advertisement with content that corresponds to their emotions, and if they are interested, they will be taken directly to a page with more information or to a purchase page.

[1076] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements based on their emotions, which will enable the provision of fast and accurate information in an aging society or during disasters, as well as further improving advertising effectiveness.

[1077] The processing flow will be explained below.

[1078] Providing real-time news

[1079] Step 1:

[1080] The server collects news data in real time from multiple news sources such as news APIs, RSS feeds, and news websites.

[1081] Step 2:

[1082] The news data collected by the server is analyzed using generative AI to identify important keywords and events.

[1083] Step 3:

[1084] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[1085] Step 4:

[1086] The server customizes news content based on the user's emotional data, adjusting the tone and content to match specific emotions.

[1087] Step 5:

[1088] The server generates customized news data as XR video using tools such as Unity and Unreal Engine.

[1089] Step 6:

[1090] The server sends the generated customized XR video data to the user's device.

[1091] Step 7:

[1092] The device displays the received customized XR video data, and the user watches it to experience customized news tailored to their emotions in an immersive format.

[1093] Providing virtual sports experiences

[1094] Step 1:

[1095] The server collects live sports data from sports APIs, real-time data feeds, etc.

[1096] Step 2:

[1097] The server uses generated AI to analyze the collected sports data and extract important highlights and key points from the game.

[1098] Step 3:

[1099] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[1100] Step 4:

[1101] The server customizes the game footage based on the user's emotional data, adjusting visual effects and narration to match specific emotions.

[1102] Step 5:

[1103] The server generates customized virtual sports game footage using 3D modeling tools.

[1104] Step 6:

[1105] The server transmits the generated customized virtual sports video data to the user's terminal.

[1106] Step 7:

[1107] The terminal displays the received customized virtual sports video data, and the user watches the video data to experience a customized match according to their emotions in an immersive format.

[1108] Providing customized advertising

[1109] Step 1:

[1110] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[1111] Step 2:

[1112] The server uses generated AI to analyze the collected behavioral data and identify the user's interests and behavioral patterns.

[1113] Step 3:

[1114] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[1115] Step 4:

[1116] The server further customizes the optimized advertising content based on the user's emotional data, generating advertising content that corresponds to a specific emotion.

[1117] Step 5:

[1118] The server generates customized advertising data, with specific designs and messages tailored to the emotion.

[1119] Step 6:

[1120] The server transmits the generated customized advertisement data to the user's terminal.

[1121] Step 7:

[1122] The device displays the personalized ad it receives, the user watches the ad, engages with the ad with emotional content, and if interested, is taken directly to a page with more information or a purchase using a QR code or clickable element.

[1123] These processing steps effectively allow users to receive important news, virtual sports experiences, and customized advertising in real time, while the incorporation of an emotion engine provides a more personalized experience.

[1124] Example 2

[1125] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1126] In modern society, information overload, misinformation, biased reporting, and other factors make it difficult for users to quickly and accurately obtain the information they need. Furthermore, in an aging society or during disasters, providing information to individual users becomes increasingly important, but general media services have difficulty providing appropriate information based on the user's emotions and state. Furthermore, current advertising systems are one-sided and are unable to provide effective advertising that is in line with the user's emotions and interests.

[1127] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1128] In this invention, the server includes means for collecting and analyzing data from multiple information sources in real time using a generation AI and extracting important information, means for recognizing a user's emotions using an emotion engine, and means for generating XR video based on the extracted information and the recognized emotions. This enables users to receive appropriate information according to their emotions quickly and in an immersive format, which is expected to improve information provision in an aging society or during disasters, as well as further increase the effectiveness of advertising.

[1129] "Generative AI" refers to algorithms and models that use artificial intelligence technology to generate data such as text and images.

[1130] "Source" refers to the means by which information is obtained from digital sources, such as news APIs, RSS feeds, websites, etc.

[1131] "Real-time" means that data is processed and transmitted the instant it is generated or acquired.

[1132] "Emotion engine" refers to algorithms and models for analyzing and recognizing user emotions.

[1133] "XR video" refers to video formats including augmented reality (AR), virtual reality (VR), and mixed reality (MR).

[1134] "User terminal" refers to digital devices such as smartphones, tablets, and personal computers.

[1135] "Live Data" refers to dynamic data that is generated and captured in real time.

[1136] "Online behavioral data" refers to data about the history and patterns of a user's activities on the Internet.

[1137] "Customized advertising" refers to advertising that is optimized based on the interests and emotions of individual users.

[1138] This invention is a comprehensive media system that uses generative AI and an emotion engine to provide users with customized news, virtual sports experiences, and advertising. Specific embodiments of this system are described below.

[1139] News data provision

[1140] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to obtain news data in real time. The collected data undergoes text analysis using generative AI (e.g., GPT-4) to extract important keywords and events. The server then uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Based on the user's emotions, the server adjusts and customizes the content of the news data. The server then generates customized XR video using tools such as Unity or Unreal Engine. The generated XR video data is sent to the user's device, where the user can view it.

[1141] As a specific example, we will adopt a method of collecting the latest earthquake news and displaying information that will reassure users who are feeling anxious. An example of a prompt sentence would be, "Collect the latest earthquake news and display it in a format that will reassure users who are feeling anxious."

[1142] Providing virtual sports experiences

[1143] The server accesses sports APIs and real-time data feeds to obtain live sports data. The collected data is analyzed by generative AI to extract important highlights and key points from the match. The server then uses an emotion engine to recognize the user's emotions in real time. Based on this, the content of the match video is adjusted and customized. Specifically, it provides more thrilling footage to excited users. The server uses 3D modeling tools such as Blender and Maya to generate customized virtual match video. The generated video data is sent to the user's device, where the user can watch it.

[1144] As a concrete example, live soccer data is analyzed and videos that highlight goal scenes, fine plays, etc. An example of a prompt sentence is "Analyze live soccer data and generate exciting videos for excited users."

[1145] Providing customized advertising

[1146] The server collects online behavior data from cookies and web traffic. The collected data is analyzed by Generative AI to identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions in real time. Based on this, advertising content is adjusted and customized. Specifically, an excited user will be served an energetic advertisement. The server then uses Generative AI to generate a customized advertisement and sends this data to the user's device. The user views the advertisement on their device and, if interested, accesses a page for additional information or to purchase.

[1147] As a concrete example, a user's online behavior data is analyzed, and if the user is interested in a particular product, an advertisement mainly consisting of positive reviews and videos about that product is provided. An example of a prompt sentence would be, "Analyze the user's online behavior data and generate a positive advertisement for the product they are interested in."

[1148] This system allows users to receive emotionally appropriate information quickly and in an immersive format, which is expected to improve information provision in aging societies and during disasters, and further increase the effectiveness of advertising.

[1149] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1150] News data provision

[1151] Step 1: Gathering news data

[1152] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to collect news data in real time. Specifically, it obtains data in JSON format through a REST API. The input of this step is news data obtained from the sources, and the output is the collected, unparsed news data.

[1153] Step 2: Analyzing news data and extracting keywords

[1154] The server uses a generative AI (e.g., GPT-4) to analyze the collected news data. It uses a text analysis algorithm to extract important keywords and events. The input of this step is the collected news data, and the output is the extracted important keywords and event information.

[1155] Step 3: Recognizing user emotions

[1156] The server uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Specifically, it analyzes the user's past behavioral data and real-time sensor data to estimate the user's emotional state. The input for this step is the user's real-time and historical data, and the output is the user's emotional state.

[1157] Step 4: Customize your news content

[1158] The server adjusts and customizes the content of the news data based on the emotion recognition results. For example, it restructures the content by emphasizing information that gives a sense of security to a user who is feeling anxious. The inputs for this step are extracted keywords, event information, and the user's emotional state, and the output is customized news content.

[1159] Step 5: Generate XR footage

[1160] The server uses tools such as Unity or Unreal Engine to generate customized XR footage, specifically creating a 3D reproduction of the news and related visual effects. The input for this step is the customized news content, and the output is the generated XR footage.

[1161] Step 6: Send and view your customized XR footage

[1162] The server sends the generated customized XR video data to the user's device. The device displays the received video data, and the user watches it. The input to this step is the generated XR video data, and the output is the video displayed on the user's device.

[1163] Providing virtual sports experiences

[1164] Step 1: Collecting live sports data

[1165] The server accesses sports APIs and real-time data feeds to collect live sports data. Specifically, it retrieves data in JSON format through REST APIs. The input of this step is the sports data retrieved from the source, and the output is the collected, unparsed sports data.

[1166] Step 2: Analyzing sports data and extracting highlights

[1167] The server uses the generative AI to analyze the collected sports data. It uses text analysis and video analysis algorithms to extract important highlights and critical moments. The input of this step is the collected sports data, and the output is the extracted highlight information.

[1168] Step 3: Recognizing user emotions

[1169] The server uses an emotion engine to recognize the user's emotions in real time, using the user's past viewing history and real-time feedback data. The input of this step is the user's real-time and historical data, and the output is the user's emotional state.

[1170] Step 4: Customize your video content

[1171] The server adjusts and customizes the content of the game footage based on the emotion recognition results. For example, it edits the footage to be more thrilling for an excited user. The input of this step is the extracted highlight information and the user's emotional state, and the output is customized game footage.

[1172] Step 5: Generate virtual matches

[1173] The server generates customized virtual game footage using 3D modeling tools such as Blender or Maya. The input of this step is the customized game content, and the output is the generated virtual game footage data.

[1174] Step 6: Send and view your customized virtual game footage

[1175] The server sends the generated virtual game video data to the user terminal. The terminal displays the received video data, and the user watches it. The input to this step is the generated virtual game video data, and the output is the video displayed on the user terminal.

[1176] Providing customized advertising

[1177] Step 1: Collect online behavior data

[1178] The server collects online behavior data from cookies and web traffic. Specifically, it obtains the data using web analytics tools such as Google Analytics. The input of this step is the user's online behavior data, and the output is the collected raw data.

[1179] Step 2: Analyze interests and behavioral patterns

[1180] The server uses the generated AI to analyze the collected data. It uses an algorithm to identify individual user interests and behavioral patterns. The input of this step is the collected behavioral data, and the output is analyzed user interest and behavioral pattern information.

[1181] Step 3: Recognizing user emotions

[1182] The server uses an emotion engine to recognize the user's emotions in real time, using real-time data collected from the webcam and microphone. The input of this step is real-time data and historical data, and the output is the user's emotional state.

[1183] Step 4: Customize your ad content

[1184] The server adjusts and customizes the advertisement content based on the emotion recognition results. For example, if the user is excited, it will provide an energetic presentation. The input of this step is the analyzed interest information and emotional state, and the output is customized advertisement content.

[1185] Step 5: Generate a personalized ad

[1186] The server uses generative AI to generate individually optimized ads, adjusting visual effects and wording. The input for this step is the customized ad content, and the output is the generated ad data.

[1187] Step 6: Send and display personalized ads

[1188] The server sends the generated advertisement data to the user terminal. The terminal displays it, and the user watches the advertisement. The input of this step is the generated advertisement data, and the output is the advertisement displayed on the user terminal.

[1189] (Application example 2)

[1190] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1191] Conventional news, sports video, and advertising systems lack personalization based on individual users' emotions and online behavior data, limiting the effectiveness of information provision. In particular, physical stores lack the means to provide users with appropriate product recommendations and promotional information, creating a demand for real-time, emotion-based customized information delivery. A system is needed to solve these issues and improve user experience.

[1192] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1193] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing the user's emotional state, means for customizing the XR video based on the emotion analysis results, and means for providing personalized product recommendation information to the user in a store, thereby enabling the provision of customized information based on the user's emotions and online behavior data.

[1194] "Generative AI" is an artificial intelligence algorithm that generates and analyzes information from massive amounts of data.

[1195] "News source" refers to the online platform or information provider that provides news data.

[1196] "News data" refers to a collection of information reported as news, including text, images, and video.

[1197] "XR video" refers to augmented reality (AR), virtual reality (VR), and mixed reality (MR) video that combines the real world and the virtual world.

[1198] A "user terminal" is a device for receiving and displaying information, and includes smartphones, smart glasses, head-mounted displays, etc.

[1199] "Emotional state" refers to the user's current psychological feelings, such as "joy," "anger," or "sadness."

[1200] "Emotion analysis" refers to technology for recognizing and classifying a user's emotional state.

[1201] "Customization" refers to tailoring content and information to best suit individual users.

[1202] "Recommended information" refers to information about products and services that are suggested to users.

[1203] "Products" refers to goods and services sold in physical stores.

[1204] "In-store" refers to the physical location where goods and services are provided.

[1205] "Personalization" refers to the unique tailoring of information or services based on a user's individual attributes, behavior, or emotions.

[1206] "Promotional information" refers to information about sales, campaigns, events, etc. for sales promotion purposes.

[1207] This invention realizes a system that uses generative AI and an emotion engine to provide individually customized information and advertisements to users, which can improve the user's shopping experience, especially in physical stores.

[1208] System Overview:

[1209] This system uses generative AI to collect and analyze real-time news data, extract and generate important information, analyze the user's emotional state, and customize XR video and product recommendations based on the results. Finally, the customized information is sent to the user's device, allowing them to view it.

[1210] Hardware and software used:

[1211] Hardware: User devices such as smartphones, smart glasses, and head-mounted displays.

[1212] Software: Generative AI models, sentiment analysis engines, notification systems, 3D modeling tools such as Unity and Unreal Engine.

[1213] Processing Details:

[1214] 1. Data Collection:

[1215] The server collects news data in real time from multiple news sources, such as news APIs and RSS feeds, as well as users' online behavioral data and emotional states.

[1216] 2. Data Analysis:

[1217] Generative AI models are used to analyze collected news data and extract important keywords and events, as well as users' online behavior data, to identify the most relevant information for each individual user.

[1218] 3. Emotion analysis:

[1219] A sentiment analysis engine is used to analyze the user's current emotional state, which is then used to customize news and product recommendations.

[1220] 4. Information generation:

[1221] Based on the analysis results, a generative AI model is used to generate customized XR footage and recommendations, including 3D modeling using Unity or Unreal Engine.

[1222] 5. Information provision:

[1223] The generated customized information is sent to the user's device via the notification system, allowing the user to view news and product recommendations in real time according to their emotional state and interests.

[1224] Examples:

[1225] For example, consider a case where a user has a search history of "shoes" and "watches," has a history of purchasing "jeans," and is currently in a "happy" emotional state. Based on this data, the server uses an emotion analysis engine to analyze the user's emotions and generate recommendations such as "casual shoes" and "sports watches." At the same time, it notifies the user of current sales and campaign information.

[1226] Example prompt sentence:

[1227] "User ID 12345's current emotional state is 'happy', and his / her online behavior data includes 'shoes' and 'watches' in his / her search history and 'jeans' in his / her purchase history. Based on this, please generate recommended products and related promotion information for the user."

[1228] This system will enable users to receive information and advertisements in real time that are optimized to their emotions and behavior, enabling a more personalized shopping experience, especially in physical stores.

[1229] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1230] Step 1:

[1231] The server collects news data in real time from multiple news sources such as news APIs and RSS feeds. In this data collection process, it loads media data such as text, images, and videos from the news sources. The input is data from the news sources, and the output is a set of collected news data.

[1232] Step 2:

[1233] The server uses a generative AI model to analyze the news data collected and extract important keywords and events. Here, the input news data is analyzed using natural language processing technology to perform summarization and topic extraction. The output is the extracted important keywords and events.

[1234] Step 3:

[1235] The server collects the user's online behavior data and emotional state data. At this stage, it obtains the user's online behavior data, such as browsing history, search history, and purchase history, and also collects emotional state data from smart devices and sensors. The input is the user's online behavior data and emotional state data, and the output is the collected user dataset.

[1236] Step 4:

[1237] The server uses an emotion analysis engine to analyze the user's current emotional state. In this step, the emotion analysis engine classifies the user's psychological state based on the collected emotional state data. The input is the user's emotional state data, and the output is the analyzed emotional state.

[1238] Step 5:

[1239] The server uses a generative AI model to generate customized XR videos and recommendations based on the emotion analysis results and online behavior data. In this step, the generative AI creates content tailored to the user's specific needs and interests. The input is the emotion analysis results and online behavior data, and the output is customized XR videos and product recommendations.

[1240] Step 6:

[1241] The server uses 3D modeling tools such as Unity or Unreal Engine to specifically render the generated customized XR video. Here, the input is the generated customized XR video data, and the output is the actual viewable XR video.

[1242] Step 7:

[1243] The server sends the generated customized XR video and product recommendation information to the user's device. In this step, the server transfers the data via the notification system, and the user receives it. The input is the customized XR video and product recommendation information, and the output is the data sent to the user's device.

[1244] Step 8:

[1245] The device displays the customized XR video and product recommendation information received to the user. The user views the video in real time through the user's device display or smart glasses. The input is the customized data sent to the user's device, and the output is the visual display to the user.

[1246] This processing step allows users to receive optimized information and advertisements in real time based on their emotional and behavioral data.

[1247] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1248] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1249] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1250] [Fourth embodiment]

[1251] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1252] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1253] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1254] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1255] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1256] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1257] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1258] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1259] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1260] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1261] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1262] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1263] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1264] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI, and also provides customized advertisements based on the user's online behavior and preferences. The specific operation form and program processing of this system are explained below.

[1265] Providing real-time news

[1266] The server collects news data from multiple news sources (APIs, RSS feeds, news websites, etc.). The server then uses generative AI to analyze the collected data and extract key information. The server generates XR video based on the extracted information and sends the data to the user's device. The device visualizes the received XR video data, and the user watches the XR video.

[1267] Specific examples

[1268] 1. The server retrieves news data through the news API.

[1269] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[1270] 3. The server uses tools such as Unity or Unreal Engine to generate important information as XR images.

[1271] 4. The server sends the generated XR video data to the user's device, which then displays the data, allowing the user to watch the news in an immersive manner.

[1272] Providing virtual sports experiences

[1273] The server collects live sports data (APIs, sports websites, real-time data feeds, etc.). The server then uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. The server generates a virtual sports game video based on the extracted data and sends the data to the user's device. The device visualizes the received virtual sports video, allowing the user to watch the game.

[1274] Specific examples

[1275] 1. The server retrieves live match data through the sports API.

[1276] 2. The server analyzes the match using a generated AI and extracts important highlights.

[1277] 3. The server uses 3D modeling tools to virtually generate footage of the match.

[1278] 4. The server sends the generated virtual sports video data to the user's device, which then displays this data, allowing the user to experience an immersive game.

[1279] Providing customized advertising

[1280] The server collects data about a user's online behavior and preferences from cookies and web traffic. The server then uses generative AI to analyze the collected data and generate ads optimized for individual users. The server then sends the customized ads it generates to the user's device, which displays them. The user views the ads and, if interested, accesses a page for additional information or a purchase.

[1281] Specific examples

[1282] 1. The server uses web analysis tools to collect user behavior data.

[1283] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[1284] 3. The server then generates a personalized ad based on this information.

[1285] 4. The server sends the generated advertising data to the user's device, which displays the advertisement, allowing the user to access the advertisement and, if interested, go directly to a page for more information or to purchase.

[1286] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements, thereby realizing fast and accurate information provision and improved advertising effectiveness in an aging society or during disasters.

[1287] The processing flow will be explained below.

[1288] Providing real-time news

[1289] Step 1:

[1290] The server collects the latest news data from multiple news sources (e.g., news APIs, RSS feeds, news websites).

[1291] Step 2:

[1292] The server uses generated AI to analyze the collected news data and extract important keywords and events, specifically using natural language processing (NLP) algorithms.

[1293] Step 3:

[1294] Based on the key information extracted by the server, XR images (e.g., 3D models and infographics) are generated using engines such as Unity or Unreal Engine.

[1295] Step 4:

[1296] The server sends the generated XR video data to the user's device.

[1297] Step 5:

[1298] The device displays the received XR video data, allowing users to watch and experience the news in an immersive format.

[1299] Providing virtual sports experiences

[1300] Step 1:

[1301] A server collects live sports data (e.g., sports API, real-time data feeds).

[1302] Step 2:

[1303] The server uses generative AI to analyze the collected sports data and extract important highlights and key points from the match, such as scores, player movements, and interesting scenes.

[1304] Step 3:

[1305] The server generates a virtually recreated sports game video based on the extracted data, again using engines such as Unity and Unreal Engine.

[1306] Step 4:

[1307] The server transmits the generated virtual sports video data to the user's terminal.

[1308] Step 5:

[1309] The device displays the received virtual sports video data, allowing users to watch it and experience it as if they were watching a real game.

[1310] Providing customized advertising

[1311] Step 1:

[1312] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[1313] Step 2:

[1314] The server uses generated AI to analyze the collected data and identify individual users' interests and behavioral patterns, specifically by analyzing their browsing history and click patterns.

[1315] Step 3:

[1316] Based on the analysis results, the server generates individually optimized advertisements, which are customized to the user's interests.

[1317] Step 4:

[1318] The server transmits the generated customized advertisement to the user's terminal.

[1319] Step 5:

[1320] The device displays the received advertisement, and if the user is interested, they can use a QR code or clickable element to access more information or a purchase page.

[1321] Through these processing steps, users can efficiently receive important news, virtual sports experiences, and customized advertising in real time.

[1322] Example 1

[1323] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1324] There is a growing need for fast and accurate information provision in an aging society and during disasters. There is also a demand for customized advertising that meets individual user needs. Conventional methods are difficult to effectively address these challenges, so a system that can provide highly accurate information in real time and a customized experience is needed.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1326] In this invention, the server includes means for collecting data in real time from multiple information sources using a generation AI and analyzing and extracting important information, means for generating mixed reality video based on the extracted information, and means for transmitting the generated mixed reality video to a user terminal and allowing the user to view it. This allows important information to be provided quickly in real time in an immersive format, making it possible to provide information in an aging society or during disasters.

[1327] "Generative AI" is an artificial intelligence technology that uses techniques such as machine learning and deep learning to generate and analyze data and extract specific information.

[1328] "Source" refers to the original media or platform that provides the data, such as an API, RSS feed, news website, or sports data feed.

[1329] "Real-time" refers to a situation in which data is generated, acquired, and analyzed almost simultaneously with real time, with extremely little delay.

[1330] "Data collection" is the process by which the server obtains data from external sources and stores it in its database.

[1331] "Analysis" is the process of processing collected data using generative AI to extract important information.

[1332] "Important information" refers to keywords, events, or phenomena that are particularly noteworthy among the collected data.

[1333] "Mixed reality video" refers to video content that combines the real world and the virtual world, generated using virtual reality (VR) or augmented reality (AR) technology.

[1334] "User device" refers to electronic devices used by users, such as computers, smartphones, tablets, and VR headsets.

[1335] "To allow a user to view" refers to the act of making a video or content available for a user to view through a user terminal.

[1336] "Live event data" refers to data that includes information about an event that is unfolding in real time, such as a sports game or a concert.

[1337] A "highlight" is a particularly important or noticeable part of an event or piece of data.

[1338] "Virtually recreated event footage" refers to footage that is virtually generated based on an actual event using 3D modeling and animation.

[1339] "Online behavior" refers to all the activities a user engages in on the Internet, including browsing, clicking, and purchasing history.

[1340] "Customized advertising" refers to advertising generated based on a user's individual interests and behavioral patterns.

[1341] This invention is a system that uses generative AI to collect data from multiple sources in real time, extract and analyze important information, and provide it to users. The system of the present invention provides visual mixed reality images to the user terminal.

[1342] Specific system configuration

[1343] Data collection

[1344] A server collects news and live event data in real time from multiple sources (APIs, RSS feeds, news websites, etc.) In this process, the server periodically sends HTTP requests to API endpoints and receives data in JSON format.

[1345] Data analysis and key information extraction

[1346] The server analyzes the collected data using a generative AI model. This analysis automatically extracts important keywords and events. For example, by providing the generative AI model with a prompt such as "Analyze the news data and extract important information," the server obtains the AI's analysis results.

[1347] Mixed reality image generation

[1348] The server uses the extracted information to generate mixed reality images using 3D modeling tools such as Unity or Unreal Engine. For example, a prompt such as "simulate a news event" can be sent to Unity, and a 3D scene can be constructed based on the simulation results.

[1349] Video data transmission

[1350] The server sends the generated mixed reality video data to the user's device using a network protocol (e.g., WebSocket or HTTP / 2) to stream the video data with low latency.

[1351] Visualization of video data

[1352] The device visualizes the received mixed reality video data and allows the user to view it. A dedicated application runs on the device, decodes the received data, and displays it on a VR headset or AR display.

[1353] Specific examples

[1354] News data provision

[1355] 1. The server retrieves news data through the news API from a URL such as "https: / / newsapi.org / v2 / top-headlines?country=jp&apiKey=YOUR_API_KEY".

[1356] 2. The server uses the generated AI to analyze the data with the prompt, "Analyze the news data and extract important information."

[1357] 3. The server uses Unity to generate a 3D scene based on a "simulated news event."

[1358] 4. The server sends the generated XR video data to the user's device.

[1359] 5. The device visualizes the received data, and the user watches an immersive news video on a VR headset.

[1360] Providing virtual sports experiences

[1361] 1. The server retrieves live match data through the sports API.

[1362] 2. The server uses a generative AI to analyze the data with the prompt, "Extract key highlights from the match data."

[1363] 3. The server uses a 3D modeling tool to virtually generate a "reenactment of the goal scene."

[1364] 4. The server sends the generated virtual sports video data to the user's device.

[1365] 5. The device visualizes the received data, allowing the user to watch an immersive game video.

[1366] Providing customized advertising

[1367] 1. The server uses web analysis tools to collect user behavior data.

[1368] 2. The server uses the generation AI to analyze the data with the prompt, "Analyze user behavior data and generate ads based on it."

[1369] 3. The server uses generated AI to generate individually customized ads.

[1370] 4. The server sends the generated advertisement to the user's device.

[1371] 5. The device displays the received advertisement, and if the user is interested, they can access the page for more information or to purchase.

[1372] In this way, the present invention provides a system that allows for fast and accurate information delivery and a customized experience.

[1373] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1374] Providing real-time news

[1375] Processing Steps

[1376] Step 1:

[1377] The server collects news data from multiple sources (APIs, RSS feeds, news websites, etc.). Specifically, the server periodically sends HTTP requests to API endpoints to retrieve data in JSON format. (Input) News API endpoint URL, (Output) JSON-formatted news data.

[1378] Step 2:

[1379] The server uses a generative AI model to analyze the collected news data. It uses the collected JSON data as input and provides the prompt, "Analyze the news data and extract important information," to the generative AI model, which then extracts important keywords and event information. (Input) JSON-formatted news data, prompt, (Output) Analysis results including important keywords and event information.

[1380] Step 3:

[1381] The server uses 3D modeling tools such as Unity or Unreal Engine to generate XR video based on the analysis results. The 3D modeling tool is given a prompt statement, "simulation of a news event," and a 3D scene is constructed based on key information. (Input) Analysis results, prompt statement, (Output) Generated XR video data.

[1382] Step 4:

[1383] The server sends the generated XR video data to the user's device. The server uses network protocols such as WebSocket or HTTP / 2 to stream the data with low latency. (Input) Generated XR video data. (Output) Transmission to the user's device is complete.

[1384] Step 5:

[1385] The device visualizes the received XR video data and the user views it. A dedicated application on the device decodes the data and displays the immersive video on a VR headset or AR display. (Input) Received XR video data, (Output) Visualized video (viewed by the user).

[1386] Providing virtual sports experiences

[1387] Processing Steps

[1388] Step 1:

[1389] The server uses the Sports API to collect live match data. The server sends a request to the API endpoint to retrieve data such as match scores and player movements in JSON format. (Input) Sports API endpoint URL, (Output) Live match data in JSON format.

[1390] Step 2:

[1391] The server uses a generative AI model to analyze live match data and extract important highlights. The generative AI model is given a prompt statement, "Please extract key highlights from the match data." The input is the live match data in JSON format, the prompt statement, and the output is the analysis results including important highlight information.

[1392] Step 3:

[1393] The server uses a 3D modeling tool to generate virtual sports footage based on the extracted highlight information. The 3D modeling tool is given the prompt "Reproduce the goal scene" to recreate the game scene. (Input) Highlight information, prompt, (Output) Generated virtual sports footage data.

[1394] Step 4:

[1395] The server sends the generated virtual sports video data to the user device. The server uses WebSocket to stream the data to the user device with low latency. (Input) Generated virtual sports video data, (Output) Transmission to the user device completed.

[1396] Step 5:

[1397] The device visualizes the received virtual sports video data and the user watches it. A dedicated application decodes the data, allowing the user to watch the game immersively using a VR headset. (Input) Received virtual sports video data, (Output) Visualized video (user viewing).

[1398] Providing customized advertising

[1399] Processing Steps

[1400] Step 1:

[1401] The server uses web analysis tools to collect data on users' online behavior. It collects cookies and web traffic data and logs users' browsing history and click behavior. (Input) User access logs and cookies, (Output) User online behavior data.

[1402] Step 2:

[1403] The server uses a generative AI model to analyze the behavioral data and determine the user's interests and behavioral patterns. It then provides the generative AI model with a prompt: "Analyze the user's behavioral data and generate advertisements based on it." (Input) The user's online behavioral data, the prompt, and (Output) the analysis results, including appropriate customized advertisement information.

[1404] Step 3:

[1405] The server uses the generative AI model to generate individually customized ads. It generates ads based on the analysis results and designs advertising campaigns. (Input) Analysis results, prompt text, (Output) Generated customized ads.

[1406] Step 4:

[1407] The server sends the generated customized advertisement to the user terminal. The server sends the advertisement data to the user terminal using the HTTP protocol. (Input) Generated customized advertisement, (Output) Transmission to the user terminal completed.

[1408] Step 5:

[1409] The device displays the received customized advertisement and the user views it. The advertisement is displayed in a browser or an application dedicated to displaying advertisements, allowing the user to access advertisements that interest them. (Input) Received customized advertisement, (Output) Displayed advertisement (viewed by the user).

[1410] (Application example 1)

[1411] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1412] There is a need for systems that can provide fast and accurate information in an aging society or during disasters. There is also a need for effective means of providing information so that security guards can patrol efficiently. While satisfying these requirements, there is also a need for systems that can display customized advertisements according to user preferences.

[1413] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1414] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing security-related information using the generation AI and customizing and providing optimal patrol routes and warning information, and means for displaying the customized information on the smart glasses, thereby enabling prompt and accurate information provision and efficient security management.

[1415] "Generative AI" is a system that uses artificial intelligence techniques to generate, analyze, and learn from data.

[1416] "News source" refers to an API, RSS feed, news website, or other source that provides news data.

[1417] "News Data" refers to articles, reports, footage, and other news-related information collected from news sources.

[1418] "Important information" refers to information such as key points, important keywords, and events extracted from news data.

[1419] "XR video" refers to video generated using technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[1420] "User terminal" refers to a device used by a user, such as a smartphone, tablet, smart glasses, or head-mounted display.

[1421] "Security-related information" refers to patrol routes, warnings, alarm information, etc. required by security guards.

[1422] A "patrol route" refers to the optimal route for a security guard to patrol.

[1423] "Warning information" refers to data that provides information on where particular caution should be exercised in a particular situation or location.

[1424] "Smart glasses" refers to a glasses-type device equipped with a display for displaying information.

[1425] "Customized advertising" refers to advertising content that is optimized based on a user's online behavior and preferences.

[1426] In this invention, the following system configuration is adopted to realize an application for smart glasses specialized for security services.

[1427] First, the server collects news data in real time from news sources, such as news APIs, RSS feeds, and news websites. Then, the server uses generative AI to analyze the collected news data and extract important information. A possible generative AI model for this purpose is OpenAI GPT-4.

[1428] Based on the extracted important information, the server generates XR images. It is recommended to use a 3D modeling tool such as Unity or Unreal Engine. The generated XR images are then sent to the user's device, such as smart glasses.

[1429] The server also analyzes security-related information in real time. It uses AI to analyze patrol routes and warning information and provides customized information. This information is also displayed on the smart glasses. This allows security guards to quickly obtain information on optimal patrol routes and important points to pay attention to.

[1430] In addition, the server collects and analyzes users' online behavior data and past patrol data. It uses generative AI to analyze this data and generate advertising information optimized for each individual user. The generated advertising information is then sent to the user's device and displayed on the smart glasses.

[1431] Specific examples

[1432] Security guard A wears smart glasses while on duty. The glasses display the latest earthquake information and evacuation instructions in real time as XR video. They also suggest optimal patrol routes based on past patrol data analyzed by the server. Additionally, the glasses display advertisements customized based on A's web browsing history.

[1433] Prompt Sentence Examples

[1434] Extract important information from the following news data: {News Data}

[1435] This invention allows security guards to take action quickly and accurately, realizing efficient security management. In addition, by displaying advertisements according to user preferences, it is expected that the effectiveness of advertisements will be improved.

[1436] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1437] Step 1:

[1438] The server collects news data from news sources, specifically, news APIs, RSS feeds, news websites, etc., via API requests. The input is the URL or API endpoint of the news source, and the output is the data of the news article or report.

[1439] Step 2:

[1440] The server analyzes the collected news data using generative AI to extract important information. The input is the news data collected in the previous step, and important keywords and events are extracted by sending prompts to the generative AI model. The output is the extracted important information, which is used in subsequent steps.

[1441] Step 3:

[1442] The server generates XR video based on the extracted key information. For this purpose, it uses 3D modeling tools such as Unity or Unreal Engine. The input is the key information extracted in step 2, and the output is the generated XR video data.

[1443] Step 4:

[1444] The server sends the generated XR video data to the user device. The input is the XR video data generated in step 3, and also includes the address of the destination user device. The output is a message that the data was successfully sent to the user device.

[1445] Step 5:

[1446] The user device visualizes the XR video data received from the server and displays it to the user. The input is the XR video data sent from the server, and the output is the video displayed on a display such as smart glasses.

[1447] Step 6:

[1448] The server collects security-related information and analyzes it using generative AI. The input is security data (e.g., security logs, surveillance camera feeds), and the analysis extracts optimal patrol routes and warning information. The output is customized security information.

[1449] Step 7:

[1450] The server sends customized patrol routes and warning information to the smart glasses. The input is the security information generated in step 6, and the output is a data transmission success message to the smart glasses.

[1451] Step 8:

[1452] The smart glasses visualize customized patrol routes and warning information received from the server and display it to the user. The input is the security information sent from the server, and the output is the information displayed on the smart glasses' display.

[1453] Step 9:

[1454] The server collects the user's online behavior data and analyzes it using Generative AI. The input is the user's online behavior data (e.g., web traffic, cookies) and analyzes the user's interests and behavioral patterns. The output is advertising information optimized for each individual user.

[1455] Step 10:

[1456] The server sends the generated customized advertisement to the user terminal, where the input is the advertisement information generated in step 9, and the output is a data transmission success message to the user terminal.

[1457] Step 11:

[1458] The user terminal displays the customized advertisement received from the server. The input is the advertisement data sent from the server, and the output is the advertisement video that the user can view.

[1459] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1460] This invention is a system that provides fast and accurate information in an aging society or during disasters through a comprehensive media service of XR video using generative AI and an emotion engine, and also provides customized advertisements based on the user's online behavior and emotions. The specific operation form and program processing of this system are explained below.

[1461] Providing real-time news

[1462] The server collects news data from multiple news sources (e.g., news APIs, RSS feeds, and news websites). The server then uses generative AI to analyze the collected news data and extract important keywords and events. The server then uses an emotion engine to recognize the user's emotions and customizes the XR video of the news data based on the user's emotions. This ensures that when the video is sent to the user's device, it contains content that corresponds to the user's specific emotions. The device displays this customized XR video data, allowing the user to watch the news.

[1463] Specific examples

[1464] 1. The server retrieves news data through the news API.

[1465] 2. The server uses generative AI to analyze and extract important keywords and events from the news data.

[1466] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the news content.

[1467] 4. The server generates customized XR footage using tools such as Unity or Unreal Engine.

[1468] 5. The server sends the customized XR video data generated to the user's device, which then displays it, allowing the user to watch news content tailored to their emotions in an immersive format.

[1469] Providing virtual sports experiences

[1470] The server collects live sports data (e.g., sports APIs, real-time data feeds). Then, the server uses generative AI to analyze the collected sports data and extract key highlights and key points of the game. Furthermore, the server uses an emotion engine to recognize the user's emotions and customize the virtual sports game video based on the user's emotions. The device displays this customized virtual sports video for the user to watch.

[1471] Specific examples

[1472] 1. The server retrieves live match data through the sports API.

[1473] 2. The server analyzes the match using a generated AI and extracts important highlights.

[1474] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the content of the game footage.

[1475] 4. The server uses 3D modeling tools to generate a customized virtual match.

[1476] 5. The server sends the generated customized virtual sports video data to the user's device, which then displays it, allowing the user to experience a match that matches their emotions in an immersive format.

[1477] Providing customized advertising

[1478] The server collects users' online behavior data from cookies and web traffic. The server then uses generative AI to analyze the collected data and identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions and further customize the advertising content. The server then sends the customized advertisements it generates to the user's device, which displays them. The user views the advertisements and, if interested, accesses a page for additional information or to purchase.

[1479] Specific examples

[1480] 1. The server uses web analysis tools to collect user behavior data.

[1481] 2. The server uses the generated AI to analyze the user's interests and behavioral patterns.

[1482] 3. The server analyzes the user's emotions using an emotion engine and adjusts and customizes the advertising content.

[1483] 4. The server then generates a personalized ad based on this information.

[1484] 5. The server sends the generated advertising data to the user's device, which displays it, allowing the user to access an advertisement with content that corresponds to their emotions, and if they are interested, they will be taken directly to a page with more information or to a purchase page.

[1485] This system allows users to receive important news in real time in an immersive format, and also allows them to watch virtual sports experiences and individually optimized advertisements based on their emotions, which will enable the provision of fast and accurate information in an aging society or during disasters, as well as further improving advertising effectiveness.

[1486] The processing flow will be explained below.

[1487] Providing real-time news

[1488] Step 1:

[1489] The server collects news data in real time from multiple news sources such as news APIs, RSS feeds, and news websites.

[1490] Step 2:

[1491] The news data collected by the server is analyzed using generative AI to identify important keywords and events.

[1492] Step 3:

[1493] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[1494] Step 4:

[1495] The server customizes news content based on the user's emotional data, adjusting the tone and content to match specific emotions.

[1496] Step 5:

[1497] The server generates customized news data as XR video using tools such as Unity and Unreal Engine.

[1498] Step 6:

[1499] The server sends the generated customized XR video data to the user's device.

[1500] Step 7:

[1501] The device displays the received customized XR video data, and the user watches it to experience customized news tailored to their emotions in an immersive format.

[1502] Providing virtual sports experiences

[1503] Step 1:

[1504] The server collects live sports data from sports APIs, real-time data feeds, etc.

[1505] Step 2:

[1506] The server uses generated AI to analyze the collected sports data and extract important highlights and key points from the game.

[1507] Step 3:

[1508] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[1509] Step 4:

[1510] The server customizes the game footage based on the user's emotional data, adjusting visual effects and narration to match specific emotions.

[1511] Step 5:

[1512] The server generates customized virtual sports game footage using 3D modeling tools.

[1513] Step 6:

[1514] The server transmits the generated customized virtual sports video data to the user's terminal.

[1515] Step 7:

[1516] The terminal displays the received customized virtual sports video data, and the user watches the video data to experience a customized match according to their emotions in an immersive format.

[1517] Providing customized advertising

[1518] Step 1:

[1519] The server uses cookies and web traffic analysis tools to collect data about your online behavior.

[1520] Step 2:

[1521] The server uses generated AI to analyze the collected behavioral data and identify the user's interests and behavioral patterns.

[1522] Step 3:

[1523] The server uses an emotion engine to recognize the user's emotion data, which is analyzed based on the video, audio, and input data received from the device.

[1524] Step 4:

[1525] The server further customizes the optimized advertising content based on the user's emotional data, generating advertising content that corresponds to a specific emotion.

[1526] Step 5:

[1527] The server generates customized advertising data, with specific designs and messages tailored to the emotion.

[1528] Step 6:

[1529] The server transmits the generated customized advertisement data to the user's terminal.

[1530] Step 7:

[1531] The device displays the personalized ad it receives, the user watches the ad, engages with the ad with emotional content, and if interested, is taken directly to a page with more information or a purchase using a QR code or clickable element.

[1532] These processing steps effectively allow users to receive important news, virtual sports experiences, and customized advertising in real time, while the incorporation of an emotion engine provides a more personalized experience.

[1533] Example 2

[1534] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1535] In modern society, information overload, misinformation, biased reporting, and other factors make it difficult for users to quickly and accurately obtain the information they need. Furthermore, in an aging society or during disasters, providing information to individual users becomes increasingly important, but general media services have difficulty providing appropriate information based on the user's emotions and state. Furthermore, current advertising systems are one-sided and are unable to provide effective advertising that is in line with the user's emotions and interests.

[1536] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1537] In this invention, the server includes means for collecting and analyzing data from multiple information sources in real time using a generation AI and extracting important information, means for recognizing a user's emotions using an emotion engine, and means for generating XR video based on the extracted information and the recognized emotions. This enables users to receive appropriate information according to their emotions quickly and in an immersive format, which is expected to improve information provision in an aging society or during disasters, as well as further increase the effectiveness of advertising.

[1538] "Generative AI" refers to algorithms and models that use artificial intelligence technology to generate data such as text and images.

[1539] "Source" refers to the means by which information is obtained from digital sources, such as news APIs, RSS feeds, websites, etc.

[1540] "Real-time" means that data is processed and transmitted the instant it is generated or acquired.

[1541] "Emotion engine" refers to algorithms and models for analyzing and recognizing user emotions.

[1542] "XR video" refers to video formats including augmented reality (AR), virtual reality (VR), and mixed reality (MR).

[1543] "User terminal" refers to digital devices such as smartphones, tablets, and personal computers.

[1544] "Live Data" refers to dynamic data that is generated and captured in real time.

[1545] "Online behavioral data" refers to data about the history and patterns of a user's activities on the Internet.

[1546] "Customized advertising" refers to advertising that is optimized based on the interests and emotions of individual users.

[1547] This invention is a comprehensive media system that uses generative AI and an emotion engine to provide users with customized news, virtual sports experiences, and advertising. Specific embodiments of this system are described below.

[1548] News data provision

[1549] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to obtain news data in real time. The collected data undergoes text analysis using generative AI (e.g., GPT-4) to extract important keywords and events. The server then uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Based on the user's emotions, the server adjusts and customizes the content of the news data. The server then generates customized XR video using tools such as Unity or Unreal Engine. The generated XR video data is sent to the user's device, where the user can view it.

[1550] As a specific example, we will adopt a method of collecting the latest earthquake news and displaying information that will reassure users who are feeling anxious. An example of a prompt sentence would be, "Collect the latest earthquake news and display it in a format that will reassure users who are feeling anxious."

[1551] Providing virtual sports experiences

[1552] The server accesses sports APIs and real-time data feeds to obtain live sports data. The collected data is analyzed by generative AI to extract important highlights and key points from the match. The server then uses an emotion engine to recognize the user's emotions in real time. Based on this, the content of the match video is adjusted and customized. Specifically, it provides more thrilling footage to excited users. The server uses 3D modeling tools such as Blender and Maya to generate customized virtual match video. The generated video data is sent to the user's device, where the user can watch it.

[1553] As a concrete example, live soccer data is analyzed and videos that highlight goal scenes, fine plays, etc. An example of a prompt sentence is "Analyze live soccer data and generate exciting videos for excited users."

[1554] Providing customized advertising

[1555] The server collects online behavior data from cookies and web traffic. The collected data is analyzed by Generative AI to identify individual users' interests and behavioral patterns. The server then uses an emotion engine to recognize users' emotions in real time. Based on this, advertising content is adjusted and customized. Specifically, an excited user will be served an energetic advertisement. The server then uses Generative AI to generate a customized advertisement and sends this data to the user's device. The user views the advertisement on their device and, if interested, accesses a page for additional information or to purchase.

[1556] As a concrete example, a user's online behavior data is analyzed, and if the user is interested in a particular product, an advertisement mainly consisting of positive reviews and videos about that product is provided. An example of a prompt sentence would be, "Analyze the user's online behavior data and generate a positive advertisement for the product they are interested in."

[1557] This system allows users to receive emotionally appropriate information quickly and in an immersive format, which is expected to improve information provision in aging societies and during disasters, and further increase the effectiveness of advertising.

[1558] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1559] News data provision

[1560] Step 1: Gathering news data

[1561] The server accesses multiple sources, such as news APIs, RSS feeds, and news websites, to collect news data in real time. Specifically, it obtains data in JSON format through a REST API. The input of this step is news data obtained from the sources, and the output is the collected, unparsed news data.

[1562] Step 2: Analyzing news data and extracting keywords

[1563] The server uses a generative AI (e.g., GPT-4) to analyze the collected news data. It uses a text analysis algorithm to extract important keywords and events. The input of this step is the collected news data, and the output is the extracted important keywords and event information.

[1564] Step 3: Recognizing user emotions

[1565] The server uses an emotion engine (e.g., Affectiva or IBM Watson) to recognize the user's emotions. Specifically, it analyzes the user's past behavioral data and real-time sensor data to estimate the user's emotional state. The input for this step is the user's real-time and historical data, and the output is the user's emotional state.

[1566] Step 4: Customize your news content

[1567] The server adjusts and customizes the content of the news data based on the emotion recognition results. For example, it restructures the content by emphasizing information that gives a sense of security to a user who is feeling anxious. The inputs for this step are extracted keywords, event information, and the user's emotional state, and the output is customized news content.

[1568] Step 5: Generate XR footage

[1569] The server uses tools such as Unity or Unreal Engine to generate customized XR footage, specifically creating a 3D reproduction of the news and related visual effects. The input for this step is the customized news content, and the output is the generated XR footage.

[1570] Step 6: Send and view your customized XR footage

[1571] The server sends the generated customized XR video data to the user's device. The device displays the received video data, and the user watches it. The input to this step is the generated XR video data, and the output is the video displayed on the user's device.

[1572] Providing virtual sports experiences

[1573] Step 1: Collecting live sports data

[1574] The server accesses sports APIs and real-time data feeds to collect live sports data. Specifically, it retrieves data in JSON format through REST APIs. The input of this step is the sports data retrieved from the source, and the output is the collected, unparsed sports data.

[1575] Step 2: Analyzing sports data and extracting highlights

[1576] The server uses the generative AI to analyze the collected sports data. It uses text analysis and video analysis algorithms to extract important highlights and critical moments. The input of this step is the collected sports data, and the output is the extracted highlight information.

[1577] Step 3: Recognizing user emotions

[1578] The server uses an emotion engine to recognize the user's emotions in real time, using the user's past viewing history and real-time feedback data. The input of this step is the user's real-time and historical data, and the output is the user's emotional state.

[1579] Step 4: Customize your video content

[1580] The server adjusts and customizes the content of the game footage based on the emotion recognition results. For example, it edits the footage to be more thrilling for an excited user. The input of this step is the extracted highlight information and the user's emotional state, and the output is customized game footage.

[1581] Step 5: Generate virtual matches

[1582] The server generates customized virtual game footage using 3D modeling tools such as Blender or Maya. The input of this step is the customized game content, and the output is the generated virtual game footage data.

[1583] Step 6: Send and view your customized virtual game footage

[1584] The server sends the generated virtual game video data to the user terminal. The terminal displays the received video data, and the user watches it. The input to this step is the generated virtual game video data, and the output is the video displayed on the user terminal.

[1585] Providing customized advertising

[1586] Step 1: Collect online behavior data

[1587] The server collects online behavior data from cookies and web traffic. Specifically, it obtains the data using web analytics tools such as Google Analytics. The input of this step is the user's online behavior data, and the output is the collected raw data.

[1588] Step 2: Analyze interests and behavioral patterns

[1589] The server uses the generated AI to analyze the collected data. It uses an algorithm to identify individual user interests and behavioral patterns. The input of this step is the collected behavioral data, and the output is analyzed user interest and behavioral pattern information.

[1590] Step 3: Recognizing user emotions

[1591] The server uses an emotion engine to recognize the user's emotions in real time, using real-time data collected from the webcam and microphone. The input of this step is real-time data and historical data, and the output is the user's emotional state.

[1592] Step 4: Customize your ad content

[1593] The server adjusts and customizes the advertisement content based on the emotion recognition results. For example, if the user is excited, it will provide an energetic presentation. The input of this step is the analyzed interest information and emotional state, and the output is customized advertisement content.

[1594] Step 5: Generate a personalized ad

[1595] The server uses generative AI to generate individually optimized ads, adjusting visual effects and wording. The input for this step is the customized ad content, and the output is the generated ad data.

[1596] Step 6: Send and display personalized ads

[1597] The server sends the generated advertisement data to the user terminal. The terminal displays it, and the user watches the advertisement. The input of this step is the generated advertisement data, and the output is the advertisement displayed on the user terminal.

[1598] (Application example 2)

[1599] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1600] Conventional news, sports video, and advertising systems lack personalization based on individual users' emotions and online behavior data, limiting the effectiveness of information provision. In particular, physical stores lack the means to provide users with appropriate product recommendations and promotional information, creating a demand for real-time, emotion-based customized information delivery. A system is needed to solve these issues and improve user experience.

[1601] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1602] In this invention, the server includes means for collecting and analyzing news data from multiple news sources in real time using a generation AI and extracting important information, means for generating XR video based on the extracted information, means for transmitting the generated XR video to a user terminal and allowing the user to view it, means for analyzing the user's emotional state, means for customizing the XR video based on the emotion analysis results, and means for providing personalized product recommendation information to the user in a store, thereby enabling the provision of customized information based on the user's emotions and online behavior data.

[1603] "Generative AI" is an artificial intelligence algorithm that generates and analyzes information from massive amounts of data.

[1604] "News source" refers to the online platform or information provider that provides news data.

[1605] "News data" refers to a collection of information reported as news, including text, images, and video.

[1606] "XR video" refers to augmented reality (AR), virtual reality (VR), and mixed reality (MR) video that combines the real world and the virtual world.

[1607] A "user terminal" is a device for receiving and displaying information, and includes smartphones, smart glasses, head-mounted displays, etc.

[1608] "Emotional state" refers to the user's current psychological feelings, such as "joy," "anger," or "sadness."

[1609] "Emotion analysis" refers to technology for recognizing and classifying a user's emotional state.

[1610] "Customization" refers to tailoring content and information to best suit individual users.

[1611] "Recommended information" refers to information about products and services that are suggested to users.

[1612] "Products" refers to goods and services sold in physical stores.

[1613] "In-store" refers to the physical location where goods and services are provided.

[1614] "Personalization" refers to the unique tailoring of information or services based on a user's individual attributes, behavior, or emotions.

[1615] "Promotional information" refers to information about sales, campaigns, events, etc. for sales promotion purposes.

[1616] This invention realizes a system that uses generative AI and an emotion engine to provide individually customized information and advertisements to users, which can improve the user's shopping experience, especially in physical stores.

[1617] System Overview:

[1618] This system uses generative AI to collect and analyze real-time news data, extract and generate important information, analyze the user's emotional state, and customize XR video and product recommendations based on the results. Finally, the customized information is sent to the user's device, allowing them to view it.

[1619] Hardware and software used:

[1620] Hardware: User devices such as smartphones, smart glasses, and head-mounted displays.

[1621] Software: Generative AI models, sentiment analysis engines, notification systems, 3D modeling tools such as Unity and Unreal Engine.

[1622] Processing Details:

[1623] 1. Data Collection:

[1624] The server collects news data in real time from multiple news sources, such as news APIs and RSS feeds, as well as users' online behavioral data and emotional states.

[1625] 2. Data Analysis:

[1626] Generative AI models are used to analyze collected news data and extract important keywords and events, as well as users' online behavior data, to identify the most relevant information for each individual user.

[1627] 3. Emotion analysis:

[1628] A sentiment analysis engine is used to analyze the user's current emotional state, which is then used to customize news and product recommendations.

[1629] 4. Information generation:

[1630] Based on the analysis results, a generative AI model is used to generate customized XR footage and recommendations, including 3D modeling using Unity or Unreal Engine.

[1631] 5. Information provision:

[1632] The generated customized information is sent to the user's device via the notification system, allowing the user to view news and product recommendations in real time according to their emotional state and interests.

[1633] Examples:

[1634] For example, consider a case where a user has a search history of "shoes" and "watches," has a history of purchasing "jeans," and is currently in a "happy" emotional state. Based on this data, the server uses an emotion analysis engine to analyze the user's emotions and generate recommendations such as "casual shoes" and "sports watches." At the same time, it notifies the user of current sales and campaign information.

[1635] Example prompt sentence:

[1636] "User ID 12345's current emotional state is 'happy', and his / her online behavior data includes 'shoes' and 'watches' in his / her search history and 'jeans' in his / her purchase history. Based on this, please generate recommended products and related promotion information for the user."

[1637] This system will enable users to receive information and advertisements in real time that are optimized to their emotions and behavior, enabling a more personalized shopping experience, especially in physical stores.

[1638] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1639] Step 1:

[1640] The server collects news data in real time from multiple news sources such as news APIs and RSS feeds. In this data collection process, it loads media data such as text, images, and videos from the news sources. The input is data from the news sources, and the output is a set of collected news data.

[1641] Step 2:

[1642] The server uses a generative AI model to analyze the news data collected and extract important keywords and events. Here, the input news data is analyzed using natural language processing technology to perform summarization and topic extraction. The output is the extracted important keywords and events.

[1643] Step 3:

[1644] The server collects the user's online behavior data and emotional state data. At this stage, it obtains the user's online behavior data, such as browsing history, search history, and purchase history, and also collects emotional state data from smart devices and sensors. The input is the user's online behavior data and emotional state data, and the output is the collected user dataset.

[1645] Step 4:

[1646] The server uses an emotion analysis engine to analyze the user's current emotional state. In this step, the emotion analysis engine classifies the user's psychological state based on the collected emotional state data. The input is the user's emotional state data, and the output is the analyzed emotional state.

[1647] Step 5:

[1648] The server uses a generative AI model to generate customized XR videos and recommendations based on the emotion analysis results and online behavior data. In this step, the generative AI creates content tailored to the user's specific needs and interests. The input is the emotion analysis results and online behavior data, and the output is customized XR videos and product recommendations.

[1649] Step 6:

[1650] The server uses 3D modeling tools such as Unity or Unreal Engine to specifically render the generated customized XR video. Here, the input is the generated customized XR video data, and the output is the actual viewable XR video.

[1651] Step 7:

[1652] The server sends the generated customized XR video and product recommendation information to the user's device. In this step, the server transfers the data via the notification system, and the user receives it. The input is the customized XR video and product recommendation information, and the output is the data sent to the user's device.

[1653] Step 8:

[1654] The device displays the customized XR video and product recommendation information received to the user. The user views the video in real time through the user's device display or smart glasses. The input is the customized data sent to the user's device, and the output is the visual display to the user.

[1655] This processing step allows users to receive optimized information and advertisements in real time based on their emotional and behavioral data.

[1656] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1657] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1658] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1659] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1660] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1661] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1662] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1663] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1664] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1665] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1666] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1667] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1668] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1669] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1670] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1671] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1672] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1673] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1674] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1675] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1676] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1677] The following is further disclosed regarding the above embodiment.

[1678] (Claim 1)

[1679] A method to collect and analyze news data from multiple news sources in real time using generative AI and extract important information.

[1680] A means for generating XR images based on the extracted information;

[1681] A means for transmitting the generated XR video to a user terminal and allowing the user to view it;

[1682] A system including:

[1683] (Claim 2)

[1684] A means to collect live sports data and use generative AI to extract key highlights and key takeaways from matches;

[1685] a means for generating a virtually reproduced sports game video based on the extracted data;

[1686] a means for transmitting the generated virtual sports video to a user terminal and allowing the user to view the video;

[1687] 10. The system of claim 1, comprising:

[1688] (Claim 3)

[1689] means of collecting data about users' online behavior and preferences;

[1690] A means of analyzing the collected data using generative AI and generating advertisements optimized for individual users;

[1691] means for transmitting the generated customized advertisement to a user terminal and displaying the advertisement to the user;

[1692] 10. The system of claim 1, comprising:

[1693] "Example 1"

[1694] (Claim 1)

[1695] A means to collect data in real time from multiple sources using generative AI, and analyze and extract important information;

[1696] a means for generating a mixed reality image based on the extracted information;

[1697] a means for transmitting the generated mixed reality image to a user terminal and allowing the user to view the image;

[1698] A system including:

[1699] (Claim 2)

[1700] A means to collect live event data and use generative AI to extract key highlights and takeaways from the event;

[1701] a means for generating a virtually reproduced event video based on the extracted data;

[1702] a means for transmitting the generated virtual event video to a user terminal and allowing the user to view the video;

[1703] 10. The system of claim 1, comprising:

[1704] (Claim 3)

[1705] Collecting data about your online activities and preferences;

[1706] A means for analyzing collected data using generation AI and generating advertisements optimized for individual users;

[1707] A means for transmitting the generated customized advertisement to a user terminal and displaying it to the user;

[1708] 10. The system of claim 1, comprising:

[1709] "Application Example 1"

[1710] (Claim 1)

[1711] A method to collect and analyze news data from multiple news sources in real time using generative AI and extract important information.

[1712] A means for generating XR images based on the extracted information;

[1713] A means for transmitting the generated XR video to a user terminal and allowing the user to view it;

[1714] A means of analyzing security-related information using generative AI and providing customized optimal patrol routes and warning information;

[1715] a means for displaying the customization information on the smart glasses;

[1716] A system including:

[1717] (Claim 2)

[1718] A means to collect live sports data and use generative AI to extract key highlights and key takeaways from matches;

[1719] a means for generating a virtually reproduced sports game video based on the extracted data;

[1720] a means for transmitting the generated virtual sports video to a user terminal and allowing the user to view the video;

[1721] 10. The system of claim 1, comprising:

[1722] (Claim 3)

[1723] means of collecting data about users' online behavior and preferences;

[1724] A means of analyzing the collected data using generative AI and generating advertisements optimized for individual users;

[1725] means for transmitting the generated customized advertisement to a user terminal and displaying the advertisement to the user;

[1726] 10. The system of claim 1, comprising:

[1727] "Example 2: Combining Emotion Engines"

[1728] (Claim 1)

[1729] A means of collecting and analyzing data from multiple sources in real time using generative AI to extract key information;

[1730] means for recognizing a user's emotion using an emotion engine;

[1731] A means for generating XR video based on the extracted information and the recognized emotions;

[1732] A means for transmitting the generated XR video to a user terminal and allowing the user to view it;

[1733] A system including:

[1734] (Claim 2)

[1735] A means to collect live data and extract key takeaways using generative AI;

[1736] means for generating a virtually reproduced image based on the extracted data and the user's emotions;

[1737] means for transmitting the generated virtual video to a user terminal and allowing the user to view the video;

[1738] 10. The system of claim 1, comprising:

[1739] (Claim 3)

[1740] means of collecting data about users' online behavior and preferences;

[1741] A means for analyzing collected data and emotions using generative AI to generate advertisements optimized for individual users;

[1742] means for transmitting the generated customized advertisement to a user terminal and displaying the advertisement to the user;

[1743] 10. The system of claim 1, comprising:

[1744] "Application example 2 when combining emotion engines"

[1745] (Claim 1)

[1746] A method to collect and analyze news data from multiple news sources in real time using generative AI and extract important information.

[1747] A means for generating XR images based on the extracted information;

[1748] A means for transmitting the generated XR video to a user terminal and allowing the user to view it;

[1749] means for analyzing the emotional state of a user;

[1750] A means to customize XR video based on emotion analysis results,

[1751] A means for providing personalized product recommendation information to a user in a store;

[1752] A system including:

[1753] (Claim 2)

[1754] A means to collect live sports data and use generative AI to extract key highlights and key takeaways from matches;

[1755] a means for generating a virtually reproduced sports game video based on the extracted data;

[1756] a means for transmitting the generated virtual sports video to a user terminal and allowing the user to view the video;

[1757] means for analyzing the emotional state of a user;

[1758] A means for customizing the virtual sports video based on the emotion analysis results;

[1759] 10. The system of claim 1, comprising:

[1760] (Claim 3)

[1761] means of collecting data about users' online behavior and preferences;

[1762] A means of analyzing the collected data using generative AI and generating advertisements optimized for individual users;

[1763] means for transmitting the generated customized advertisement to a user terminal and displaying the advertisement to the user;

[1764] means for providing customized promotional information to the user in the store;

[1765] 10. The system of claim 1, comprising: [Explanation of symbols]

[1766] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting and analyzing news data from multiple news sources in real time using generative AI to extract important information; A means for generating XR images based on the extracted information; A means for transmitting the generated XR video to a user terminal and allowing the user to view it; A system including:

2. A means to collect live sports data and use generative AI to extract key highlights and key takeaways from matches; a means for generating a virtually reproduced sports game video based on the extracted data; a means for transmitting the generated virtual sports video to a user terminal and allowing the user to view the video; The system of claim 1 , comprising:

3. means of collecting data about users' online behavior and preferences; A means of analyzing the collected data using generative AI and generating advertisements optimized for individual users; means for transmitting the generated customized advertisement to a user terminal and displaying the customized advertisement to the user; The system of claim 1 , comprising:

Images