system

Abstract

We provide the system. [Solution] A means of converting information acquired using generative artificial intelligence into a virtual reality format, A means for distributing the information converted into the aforementioned virtual reality format to a user terminal, Means for inserting advertisements associated with the aforementioned virtual reality format information, A system that includes this.

Description

Technical Field

[0004] , , ,

[0005] , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional news distribution, it has been difficult to continuously attract the interest of users, and it has also been difficult for advertisers to obtain sufficient advertising effects. Furthermore, the visual flatness and one-way nature of online news have been factors that reduce the interest of users. In order to overcome such problems, it is necessary to innovate the method of presenting information and provide a more attractive and immersive experience for users.

Means for Solving the Problems

[0005] <000003This invention provides a system that uses generative artificial intelligence to convert news information acquired in real time into a virtual reality format. This system delivers the virtual reality information to the user's terminal, enabling the user to have an immersive news experience. Furthermore, by analyzing the acquired news information and extracting and presenting important information, it provides users with highly relevant content. In addition, by selecting the most suitable advertisements based on the user's profile information and naturally inserting them into the virtual reality information, it also provides advertisers with an effective promotional tool.

[0006] "Generative artificial intelligence" refers to an algorithm or system that can generate new information or content based on acquired data.

[0007] "Virtual reality" is a technology that allows users to experience immersion through sight and sound in an artificial environment created using computer technology.

[0008] A "user terminal" is a device used by a user to receive, process, and display information, and includes smartphones, tablets, VR headsets, and other similar devices.

[0009] "Advertising" refers to content that presents information to raise awareness of a specific product or service.

[0010] "Profile information" refers to data such as a user's attributes, preferences, and behavioral history, and is information about an individual user. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F manages communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] The present invention aims to convert news data into a virtual reality format using generative artificial intelligence and deliver it to user terminals. To implement this system, multiple servers work in cooperation to provide users with an immersive news experience.

[0033] First, the server retrieves the latest news data through APIs and other online sources. This raw news data is collected in various forms, including text, images, and videos. The server then analyzes this data and prioritizes the information based on its importance and relevance. This analysis includes classifying and summarizing news topics.

[0034] Next, the server uses generative artificial intelligence to convert the analyzed data into a virtual reality format. In this process, the AI ​​model generates 3D visuals and audio, visually and aurally representing the news event. When the user experiences this through a VR headset, they get a sense of being right there at the news event.

[0035] Furthermore, this system also has the ability to effectively insert advertisements. Based on the user's profile information and behavioral history, the server selects targeted advertisements and places them in appropriate locations within the generated virtual reality content. This process allows meaningful advertisements to be displayed without disrupting the user experience.

[0036] For example, consider a scenario where a user is watching news about an international sporting event. The server collects data including the latest match results and player comments, and uses AI to recreate match highlights and player interviews in VR format. This allows the user to experience the atmosphere of being at the venue, and the natural display of sports equipment advertisements can further stimulate interest in related products.

[0037] Thus, the system of the present invention is a new form that not only innovates conventional news distribution methods and provides users with an immersive experience, but also offers advertisers an effective promotional tool.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The server retrieves news data in real time from APIs and other sources. The news data is collected in text, image, and video formats.

[0041] Step 2:

[0042] The server analyzes acquired news data, classifying and summarizing the information. It identifies important topics and relevant media, and converts them into a format that can be efficiently processed by generative AI models.

[0043] Step 3:

[0044] The server uses generated artificial intelligence to convert analyzed news data into a virtual reality format. This includes generating 3D models and creating stereophonic audio, resulting in a more realistic experience.

[0045] Step 4:

[0046] The server references the user's profile information and selects highly relevant advertisements. By seamlessly integrating targeted ads with virtual reality content, a visually natural ad display is achieved.

[0047] Step 5:

[0048] The server streams integrated VR content to the user's device. The user can then experience the received virtual reality content through a VR headset.

[0049] Step 6:

[0050] Users experience news through sight and sound within a VR environment. They can explore the news from a three-dimensional perspective and interact with interactive elements.

[0051] Step 7:

[0052] The device collects user interaction data and feedback. This data is sent to a server and used to improve the system and optimize advertising in the future.

[0053] (Example 1)

[0054] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0055] In today's world, a vast amount of information is generated daily, and the challenge lies in providing it to users in a meaningful way. News content, in particular, needs to be delivered quickly and in an immersive format, but traditional methods result in a limited user experience, and challenges remain in appropriately prioritizing information and effectively inserting relevant advertisements.

[0056] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0057] In this invention, the server includes means for acquiring information from multiple information sources using an information processing device, means for analyzing the acquired information and classifying and prioritizing the information based on its importance and relevance, and means for converting the analyzed information into a virtual reality format using generative artificial intelligence. This makes it possible to provide users with a more immersive news experience while appropriately inserting relevant advertisements.

[0058] An "information processing device" is a mechanical or electronic device used for collecting, processing, and transmitting data.

[0059] "Information sources" refer to sources or platforms for providing data and knowledge, including online news sites and other digital content providers.

[0060] "Generative artificial intelligence" refers to an advanced computer program or system that can automatically generate or transform data using machine learning techniques.

[0061] "Virtual reality" refers to a digital representation format that includes three-dimensional visual and auditory elements generated by a computer, providing users with an immersive experience.

[0062] "User equipment" refers to devices used by end users to receive and experience information, and includes computers and VR headsets.

[0063] "Characteristic information" refers to data related to the attributes and behavioral history of individual users, including the user's age, gender, interests, etc.

[0064] In this embodiment of the invention, a server acquires news data from an information source using an information processing device. The server can acquire news data in various formats (text, images, videos, etc.) by utilizing APIs and web crawlers connected to the internet. The acquired data is classified and prioritized through an analysis process within the server. This analysis uses natural language processing techniques and machine learning algorithms, specifically generative AI models. As a generative AI model, for example, an existing large-scale language model can be customized and used.

[0065] The server inputs the analyzed data into a generating artificial intelligence using prompts, and then generates virtual reality content. In this process, the AI ​​model generates 3D visual and audio content and converts it into a format suitable for playback on the user's device. Specific software used includes 3D content generation platforms such as Unity and Unreal Engine.

[0066] Users can experience this virtual reality content through a VR headset or compatible display device. This device receives the VR content transmitted from the server and displays it to the user in real time. Furthermore, the server refers to the user's characteristic information and selects the most suitable advertisements to integrate into the virtual reality content. This ensures that advertisements are displayed effectively without disrupting the user experience.

[0067] A concrete example is a scenario where a user experiences news about an international sporting event in VR. In this case, the server retrieves the latest match results and uses an AI model to generate match highlights in VR format. The generated content is delivered to the user's VR headset, and advertisements for related sports equipment are displayed simultaneously. An example of a prompt that enables such an experience would be, "Generate highlights of the latest international soccer match in 3D VR format and provide audio for player interviews."

[0068] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0069] Step 1:

[0070] The server collects news data from information sources. Specifically, the server retrieves news from the internet using APIs and web crawlers. The input to the server at this time is access information such as URIs and API keys necessary for retrieving the news data. The output is raw news data in various formats, including text, images, and videos.

[0071] Step 2:

[0072] The server analyzes the collected news data. In this process, the server uses natural language processing techniques to analyze the data and evaluate the importance and relevance of the news. The input data is the collected raw news information. The output includes news topic classifications, summarized information, and their priority rankings.

[0073] Step 3:

[0074] The server generates virtual reality content using artificial intelligence based on the analyzed news data. The server formats the analysis results into prompt sentences and inputs them into the AI ​​model, giving specific instructions such as "Please represent this news in 3D VR format." The input consists of the analyzed data and prompt sentences, and the output is VR content including 3D visuals and audio.

[0075] Step 4:

[0076] The server delivers the generated virtual reality content to the user. The server streams the content data to the user's terminal via the network. The input is the generated VR content, and the output is the data sent to the user's terminal.

[0077] Step 5:

[0078] The terminal receives VR content transmitted from the server and provides it to the user. Specifically, the user wears a VR headset to access a virtual reality space and experience an immersive experience. The input is content data transmitted from the server, and the output is the visual and auditory VR experience provided to the user.

[0079] Step 6:

[0080] The server selects appropriate advertisements based on user characteristics and integrates them into the VR content. The input is the user's profile data and the generated VR content, while the output is the VR content with the advertisements inserted. This process ensures that targeted advertisements are delivered effectively without compromising the user experience.

[0081] (Application Example 1)

[0082] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0083] Traditional news distribution methods only provide information in the form of text, images, and videos, limiting the sense of immersion and presence for users. Furthermore, the advertisements delivered were not optimized based on user interests, resulting in a lack of consistency in the information experience. In addition, while there is a need to improve the information experience through the use of visual devices, appropriate technologies have not been sufficiently available.

[0084] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0085] In this invention, the server includes means for converting information acquired using generative artificial intelligence into a virtual reality format, means for delivering the information converted into the virtual reality format to the user's visual device, and means for dynamically selecting and placing advertising information associated with the information in the virtual reality format. This enables the user to receive news as an immersive experience and to be presented with advertisements based on their personal interests.

[0086] "Generative artificial intelligence" is a technology that uses machine learning models to generate new information from diverse data.

[0087] "Virtual reality format" is an information format that uses computer-generated 3D visuals and sound to simulate real-world experiences.

[0088] "User's visual device" refers to a device that a user wears or uses to receive visual digital information, and includes, for example, smart glasses and head-mounted displays.

[0089] "Advertising information" refers to notices and promotional information presented to users for commercial or public interest purposes.

[0090] An "immersive experience" is a form of experience in which users can gain a strong sense of presence, as if they were actually in the real world.

[0091] "Personal interest-based advertising" is a method of selecting and presenting advertisements that are highly relevant to a person based on their past history and attribute information.

[0092] The system implementing this invention is server-centric and features the function of converting news information acquired using generative artificial intelligence into a virtual reality format. News information is collected in various formats such as text, images, and videos, and the server analyzes them. The analysis includes topic classification and summarization using natural language processing techniques. Based on the analysis results, a generative AI model generates 3D visuals and audio, which are then delivered as news content in virtual reality format.

[0093] Through the user's visual devices, such as smart glasses or a head-mounted display, the user receives the delivered content as an immersive experience. The server processes the 3D visuals and audio using virtual reality content generation software such as Unity or Unreal Engine. This allows the user to feel as if they are actually at the news event.

[0094] Furthermore, the server analyzes the user's personal attribute information and executes an algorithm that dynamically selects relevant advertising information and incorporates it into the virtual reality content. This allows for the effective presentation of advertisements tailored to the user's interests.

[0095] As a concrete example, consider a scenario where a user is watching news about an international sporting event. The server collects the latest match results and player comments, and uses generative AI to recreate match highlights and player interviews in 3D. Then, it inputs the prompt, "What prompt would you like to use to visualize the latest sports news and player interviews?" into the generative AI model, providing the user with an immersive experience. In this process, it is possible to naturally present specific product advertisements from national brands to stimulate the user's interest in sports equipment.

[0096] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0097] Step 1:

[0098] The server uses APIs to retrieve the latest news data from external news sources. It takes URLs or API endpoints of multiple news articles as input and stores text, images, and video data as output in local storage. Specifically, the server collects the necessary data from each information source and temporarily stores it in a structured format.

[0099] Step 2:

[0100] The server analyzes the content of the acquired news data using a natural language processing model to extract important topics and keywords. Using the previously saved text data of the news articles as input, it generates topics and summaries with an evaluation of the importance of each article as output. Specifically, the server executes an analysis algorithm to prioritize news topics.

[0101] Step 3:

[0102] The server uses a generative AI model to create prompts for generating virtual reality content based on the analyzed data. It takes topic information with assigned importance levels as input and generates prompts as output. Specifically, it prepares prompts in the format, "Generate 3D visuals and sound for this topic."

[0103] Step 4:

[0104] The generative AI model constructs 3D visuals and audio content based on prompt messages received from the server. It uses prompt messages as input and generates virtual reality-style digital content as output. Specifically, it utilizes the capabilities of Unity or Unreal Engine to render scenes that provide users with a visual and auditory experience.

[0105] Step 5:

[0106] The server dynamically selects advertisements that match the user's interests, taking into account the user's attribute information, and embeds them in the generated virtual reality content. It uses user profile data as input and generates content with embedded advertisements as output. Specifically, it analyzes the user's purchase and browsing history to select relevant advertisements.

[0107] Step 6:

[0108] The user's visual device receives virtual reality content from a server, providing an immersive news experience. It takes modified 3D visual and audio data as input and presents visual and auditory information to the user as output. Specifically, the device processes the received data in real time and displays it to the user.

[0109] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0110] This invention is a system that combines generative artificial intelligence and an emotion engine, aiming to improve the user experience through a process of converting acquired news data into a virtual reality format and delivering it to the user. The system consists of several main components, including a server, a user terminal, and an emotion engine.

[0111] First, the server collects news data from multiple sources. This data is collected in various formats, such as text, images, and videos, and then analyzed by the server. Through this analysis, key points of the news are extracted and optimized for conversion into a virtual reality format.

[0112] Next, the server uses a generative artificial intelligence model to convert the news into a virtual reality format. In this process, 3D objects and virtual environments are generated based on the content of the news to enhance user immersion. In addition, an emotion engine acquires emotion data from the user's facial expressions and voice on the device and sends it to the server.

[0113] The server leverages this sentiment data to provide users with the optimal news experience. For example, if a user shows a surprised expression, the AI ​​can provide additional information or visual effects that emphasize that emotion. This dynamic adjustment personalizes the news reading experience and keeps users engaged.

[0114] Furthermore, this system is also beneficial for advertisers. The emotion engine analyzes users' emotions and adjusts ad content based on that information, enabling more targeted advertising. For example, displaying ads for specific products when users are showing positive emotions can increase the effectiveness of the ads.

[0115] Thus, the system of the present invention enriches the news experience in virtual reality while reflecting the user's emotions in real time, and also contributes to optimizing advertising effectiveness. As a specific example, when a user watches a live sports event on the news, the video and sound are dynamically adjusted in response to changes in emotions at important moments in the game, allowing the user to experience a feeling as if they were actually there.

[0116] The following describes the processing flow.

[0117] Step 1:

[0118] The server retrieves the latest news data from APIs and information sources. The news data includes text, images, and video formats.

[0119] Step 2:

[0120] The server analyzes the acquired news data, extracting keywords and performing summarization. This organizes the important information necessary for conversion into a virtual reality format.

[0121] Step 3:

[0122] The server uses artificial intelligence to generate virtual reality content based on extracted information. Specifically, it creates 3D models and audio guides to build immersive content for users.

[0123] Step 4:

[0124] The device activates an emotion engine to recognize the user's emotions through their facial expressions and voice. The emotion engine acquires the user's emotion data in real time and sends it to the server.

[0125] Step 5:

[0126] The server analyzes the emotion data it receives and dynamically adjusts the virtual reality content according to the user's emotions. For example, if the user is surprised, the visual effects are enhanced.

[0127] Step 6:

[0128] The server selects and places ads. Ads are included appropriately based on emotions to avoid disrupting the user experience.

[0129] Step 7:

[0130] The server builds the final virtual reality content and streams it to the user's device. The user can then begin an immersive news experience on their device.

[0131] Step 8:

[0132] Users experience the provided virtual reality content and view information and advertisements tailored to their emotions. Once the user experience ends, the device sends interaction data to a server, which is used for further system improvements.

[0133] (Example 2)

[0134] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0135] Traditional news distribution systems lacked sufficient personalization to consider users' emotional responses, failing to maximize the impact news content had on users' emotions. Furthermore, virtual reality-based information delivery was not effectively utilized, making it difficult for users to have an immersive experience. Additionally, advertising was based solely on general user profile information, lacking real-time targeting that considered emotional data.

[0136] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0137] In this invention, the server includes means for converting information acquired using generative artificial intelligence into a virtual reality format, means for optimizing the information converted into a virtual reality format based on the user's emotional data, and means for delivering the optimized virtual reality format information to the user's terminal. This enables personalized news delivery that reflects the user's emotions in real time, resulting in a higher level of immersion and more effective advertising.

[0138] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new information and content in a natural way based on acquired data.

[0139] "Virtual reality" refers to a three-dimensional visual and auditory environment created using digital technology, providing users with an immersive experience.

[0140] "Emotional data" refers to data that indicates a user's emotional state, extracted from biometric information such as facial expressions and voice.

[0141] A "user terminal" refers to a computing device operated by a user, which is used to display virtual reality content and acquire emotional data.

[0142] "Advertising" refers to information or content presented to users for the purpose of promoting a specific product or service.

[0143] The system for implementing this invention mainly consists of multiple components, such as a server, a terminal, and a user.

[0144] First, the server collects news data from multiple sources. This collection includes obtaining data in text, image, and video formats via APIs. The server then uses natural language processing (NLP) techniques to analyze the news data and extract important information.

[0145] Next, using a generative artificial intelligence model, the server converts the analyzed news data into a virtual reality format. Specifically, it generates 3D objects and virtual environments corresponding to the news content, providing the user with an immersive experience. For example, in news reporting on a sports match, the generative AI model recreates a virtual stadium and realistic movements of the players. For this process, the generative AI model is input with a text-based prompt, "Recreate the key moments of this match."

[0146] Meanwhile, the user's device analyzes the user's facial expressions and voice via an emotion engine, acquiring emotion data in real time. This data is sent to a server and used to optimize the virtual reality content. The server dynamically adjusts the visual and auditory elements within the virtual environment according to the emotions the user expresses, creating a personalized news experience.

[0147] Furthermore, the server selects information provided by advertisers based on user profile information and sentiment data, and displays optimized ads to the target audience. As a result, users view more relevant ads, improving advertising effectiveness.

[0148] In this way, the invention reflects user emotions in real time, enabling personalized virtual reality news delivery.

[0149] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0150] Step 1:

[0151] The server collects news data. It retrieves data in text, image, and video formats from news sources via APIs. At this stage, the input is raw data provided by each source, and the output is an integrated news dataset. Within the server, this data undergoes basic data organization processes, such as being sorted by date and category.

[0152] Step 2:

[0153] The server analyzes the collected data. It uses natural language processing (NLP) techniques to extract keywords from text data and sentiment analysis to determine the tone of the news. Image and video data are analyzed using image recognition algorithms to detect visually significant points. The input to this process is integrated news data, and the output is an analyzed dataset. Specific operations include saving the information to a database.

[0154] Step 3:

[0155] The server generates virtual reality content using a generative AI model. Based on the analyzed data, it takes the prompt "Recreate this news content in a 3D environment" as input and generates a virtual environment and 3D objects. The input for this process is the analyzed news data, and the output is digital content in virtual reality format.

[0156] Step 4:

[0157] The device collects user emotion data. Using the device's camera and microphone, it captures the user's facial expressions and voice in real time, which are then analyzed by an emotion engine. The input for this step is the user's real-time biometric information, and the output is the analyzed emotion data. This information is then transferred from the device to the server.

[0158] Step 5:

[0159] The server optimizes the virtual reality content based on the acquired emotion data. The visual effects and sound elements of the content are adjusted according to the user's emotional state. The input for this step is the user's emotion data, and the output is the optimized virtual reality content. Specifically, if the user is surprised, actions such as emphasizing color effects are performed.

[0160] (Application Example 2)

[0161] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0162] In modern society, simply providing visual information is insufficient for users to have a fully immersive experience. Furthermore, the lack of information delivery tailored to users' emotions and states, as well as the optimization of advertising, means that valuable information experiences cannot be provided to users.

[0163] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0164] In this invention, the server includes means for acquiring and analyzing user emotional data, means for converting the acquired information into a virtual reality format using generative artificial intelligence, and means for dynamically adjusting the information in the virtual reality format based on the emotional data. This makes it possible to provide an immersive information experience that is tailored to the user's emotions and preferences, thereby improving the targeting effectiveness of advertisements.

[0165] "User emotion data" refers to information indicating the emotional state of a user, obtained from their facial expressions, voice, and actions.

[0166] "Means of analysis" refers to algorithms and processes used to evaluate a user's current emotional state based on acquired emotional data.

[0167] "Generative artificial intelligence" is an AI technology used for generating virtual reality content, possessing the ability to create new content based on input data.

[0168] "Means of converting to virtual reality format" refers to the technologies and processes used to construct acquired information as 3D objects or virtual environments, thereby providing users with an immersive experience.

[0169] "Means of dynamic adjustment" refers to technologies or algorithms for changing the content and display method in real time according to the user's emotional state.

[0170] "Means of inserting advertisements" refers to methods for effectively placing advertisements related to virtual reality content and presenting information without disrupting the user experience.

[0171] The system implementing this invention consists of a server, a user terminal, and an emotion engine. The specific implementation method is described below.

[0172] The server first acquires emotional data through the user's camera and microphone, and then uses an emotion recognition API to analyze it. The emotion recognition API analyzes the user's facial expressions and voice in real time and quantifies the user's emotional state. This data is used as foundational data to personalize the user's experience.

[0173] Next, the server collects news data from various sources and converts it into a virtual reality format using a generative AI model. The generative AI model generates 3D objects and virtual environments from the collected text, images, and videos, allowing users to enjoy a visually rich and immersive experience. This process utilizes virtual reality content generation software such as Unity and Unreal Engine.

[0174] Furthermore, the user's device dynamically adjusts the virtual reality content generated by the server based on the analyzed emotional data. For example, if the user expresses positive emotions, the brightness and color tone of the video can be adjusted, or the background music can be changed to improve the user experience.

[0175] Regarding advertising, targeting is performed based on user profile information and sentiment data. This allows for the effective insertion of advertisements tailored to the user's state into the virtual reality space, maximizing the effectiveness of the advertising.

[0176] For example, when a user is watching virtual reality sports news, the generative AI model can detect the user's surprise at key moments and add special effects to highlight the footage at that time. This allows the user to feel as if they are actually there.

[0177] An example of a prompt to input into a generative AI model is: "Generate the VR scene that best suits the user's emotion based on the following news event. User reaction: surprise. Event: Analysis of speeches at an important political meeting." By using this prompt, the system can provide a personalized and immersive experience for the user.

[0178] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0179] Step 1:

[0180] The server acquires emotional data through the user's device's camera and microphone. The user's facial expressions and voice are used as input and passed to an emotion recognition API. The API analyzes this data and outputs numerical data indicating the user's emotional state. This process collects the user's current feelings and reactions as digital information.

[0181] Step 2:

[0182] The server collects news data from multiple sources. Inputs include online news feeds, relevant text, images, and videos. The server processes this raw data, extracts key news points, and prepares it for a generative AI model. This prioritizes important information and filters out other information.

[0183] Step 3:

[0184] The server uses a generative AI model to convert acquired news data into a virtual reality format. The input consists of news text and image data, which the model uses to generate 3D objects and virtual environments, creating visually meaningful content for the user. The output is VR content designed to provide an immersive experience.

[0185] Step 4:

[0186] The server dynamically adjusts the generated virtual reality content based on the analyzed emotional data. Emotional state data is used as input, and the color tone and sound effects of the video are adjusted accordingly. This creates an experience that matches the user's emotions.

[0187] Step 5:

[0188] The server combines user sentiment data and profile information to select the most suitable advertisements and insert them into the virtual reality content. The input includes user profile information and sentiment data, which the AI ​​uses to select the most appropriate advertisements and seamlessly integrate them into the content. This maximizes the effectiveness of the advertisements and creates content that captures the user's attention.

[0189] Step 6:

[0190] The user terminal displays virtual reality content delivered from the server. The terminal utilizes a VR headset or display to provide the generated content to the user visually and aurally, creating an immersive news experience. During this process, dynamically adjusted video and audio are delivered to the user, enabling a more realistic virtual reality experience.

[0191] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0192] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0193] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0194] [Second Embodiment]

[0195] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0196] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0197] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0198] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0199] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0200] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0201] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0202] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0203] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0204] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0205] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0206] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0207] The present invention aims to convert news data into a virtual reality format using generative artificial intelligence and deliver it to user terminals. To implement this system, multiple servers work in cooperation to provide users with an immersive news experience.

[0208] First, the server retrieves the latest news data through APIs and other online sources. This raw news data is collected in various forms, including text, images, and videos. The server then analyzes this data and prioritizes the information based on its importance and relevance. This analysis includes classifying and summarizing news topics.

[0209] Next, the server uses generative artificial intelligence to convert the analyzed data into a virtual reality format. In this process, the AI ​​model generates 3D visuals and audio, visually and aurally representing the news event. When the user experiences this through a VR headset, they get a sense of being right there at the news event.

[0210] Furthermore, this system also has the ability to effectively insert advertisements. Based on the user's profile information and behavioral history, the server selects targeted advertisements and places them in appropriate locations within the generated virtual reality content. This process allows meaningful advertisements to be displayed without disrupting the user experience.

[0211] For example, consider a scenario where a user is watching news about an international sporting event. The server collects data including the latest match results and player comments, and uses AI to recreate match highlights and player interviews in VR format. This allows the user to experience the atmosphere of being at the venue, and the natural display of sports equipment advertisements can further stimulate interest in related products.

[0212] Thus, the system of the present invention is a new form that not only innovates conventional news distribution methods and provides users with an immersive experience, but also offers advertisers an effective promotional tool.

[0213] The following describes the processing flow.

[0214] Step 1:

[0215] The server retrieves news data in real time from APIs and other sources. The news data is collected in text, image, and video formats.

[0216] Step 2:

[0217] The server analyzes acquired news data, classifying and summarizing the information. It identifies important topics and relevant media, and converts them into a format that can be efficiently processed by generative AI models.

[0218] Step 3:

[0219] The server uses generated artificial intelligence to convert analyzed news data into a virtual reality format. This includes generating 3D models and creating stereophonic audio, resulting in a more realistic experience.

[0220] Step 4:

[0221] The server references the user's profile information and selects highly relevant advertisements. By seamlessly integrating targeted ads with virtual reality content, a visually natural ad display is achieved.

[0222] Step 5:

[0223] The server streams integrated VR content to the user's device. The user can then experience the received virtual reality content through a VR headset.

[0224] Step 6:

[0225] Users experience news through sight and sound within a VR environment. They can explore the news from a three-dimensional perspective and interact with interactive elements.

[0226] Step 7:

[0227] The device collects user interaction data and feedback. This data is sent to a server and used to improve the system and optimize advertising in the future.

[0228] (Example 1)

[0229] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0230] In today's world, a vast amount of information is generated daily, and the challenge lies in providing it to users in a meaningful way. News content, in particular, needs to be delivered quickly and in an immersive format, but traditional methods result in a limited user experience, and challenges remain in appropriately prioritizing information and effectively inserting relevant advertisements.

[0231] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0232] In this invention, the server includes means for acquiring information from multiple information sources using an information processing device, means for analyzing the acquired information and classifying and prioritizing the information based on its importance and relevance, and means for converting the analyzed information into a virtual reality format using generative artificial intelligence. This makes it possible to provide users with a more immersive news experience while appropriately inserting relevant advertisements.

[0233] An "information processing device" is a mechanical or electronic device used for collecting, processing, and transmitting data.

[0234] "Information sources" refer to sources or platforms for providing data and knowledge, including online news sites and other digital content providers.

[0235] "Generative artificial intelligence" refers to an advanced computer program or system that can automatically generate or transform data using machine learning techniques.

[0236] "Virtual reality" refers to a digital representation format that includes three-dimensional visual and auditory elements generated by a computer, providing users with an immersive experience.

[0237] "User equipment" refers to devices used by end users to receive and experience information, and includes computers and VR headsets.

[0238] "Characteristic information" refers to data related to the attributes and behavioral history of individual users, including the user's age, gender, interests, etc.

[0239] In this embodiment of the invention, a server acquires news data from an information source using an information processing device. The server can acquire news data in various formats (text, images, videos, etc.) by utilizing APIs and web crawlers connected to the internet. The acquired data is classified and prioritized through an analysis process within the server. This analysis uses natural language processing techniques and machine learning algorithms, specifically generative AI models. As a generative AI model, for example, an existing large-scale language model can be customized and used.

[0240] The server inputs the analyzed data into a generating artificial intelligence using prompts, and then generates virtual reality content. In this process, the AI ​​model generates 3D visual and audio content and converts it into a format suitable for playback on the user's device. Specific software used includes 3D content generation platforms such as Unity and Unreal Engine.

[0241] Users can experience this virtual reality content through a VR headset or compatible display device. This device receives the VR content transmitted from the server and displays it to the user in real time. Furthermore, the server refers to the user's characteristic information and selects the most suitable advertisements to integrate into the virtual reality content. This ensures that advertisements are displayed effectively without disrupting the user experience.

[0242] A concrete example is a scenario where a user experiences news about an international sporting event in VR. In this case, the server retrieves the latest match results and uses an AI model to generate match highlights in VR format. The generated content is delivered to the user's VR headset, and advertisements for related sports equipment are displayed simultaneously. An example of a prompt that enables such an experience would be, "Generate highlights of the latest international soccer match in 3D VR format and provide audio for player interviews."

[0243] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0244] Step 1:

[0245] The server collects news data from information sources. Specifically, the server retrieves news from the internet using APIs and web crawlers. The input to the server at this time is access information such as URIs and API keys necessary for retrieving the news data. The output is raw news data in various formats, including text, images, and videos.

[0246] Step 2:

[0247] The server analyzes the collected news data. In this process, the server uses natural language processing techniques to analyze the data and evaluate the importance and relevance of the news. The input data is the collected raw news information. The output includes news topic classifications, summarized information, and their priority rankings.

[0248] Step 3:

[0249] The server generates virtual reality content using artificial intelligence based on the analyzed news data. The server formats the analysis results into prompt sentences and inputs them into the AI ​​model, giving specific instructions such as "Please represent this news in 3D VR format." The input consists of the analyzed data and prompt sentences, and the output is VR content including 3D visuals and audio.

[0250] Step 4:

[0251] The server delivers the generated virtual reality content to the user. The server streams the content data to the user's terminal via the network. The input is the generated VR content, and the output is the data sent to the user's terminal.

[0252] Step 5:

[0253] The terminal receives VR content transmitted from the server and provides it to the user. Specifically, the user wears a VR headset to access a virtual reality space and experience an immersive experience. The input is content data transmitted from the server, and the output is the visual and auditory VR experience provided to the user.

[0254] Step 6:

[0255] The server selects appropriate advertisements based on user characteristics and integrates them into the VR content. The input is the user's profile data and the generated VR content, while the output is the VR content with the advertisements inserted. This process ensures that targeted advertisements are delivered effectively without compromising the user experience.

[0256] (Application Example 1)

[0257] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0258] Traditional news distribution methods only provide information in the form of text, images, and videos, limiting the sense of immersion and presence for users. Furthermore, the advertisements delivered were not optimized based on user interests, resulting in a lack of consistency in the information experience. In addition, while there is a need to improve the information experience through the use of visual devices, appropriate technologies have not been sufficiently available.

[0259] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0260] In this invention, the server includes means for converting information acquired using generative artificial intelligence into a virtual reality format, means for delivering the information converted into the virtual reality format to the user's visual device, and means for dynamically selecting and placing advertising information associated with the information in the virtual reality format. This enables the user to receive news as an immersive experience and to be presented with advertisements based on their personal interests.

[0261] "Generative artificial intelligence" is a technology that uses machine learning models to generate new information from diverse data.

[0262] "Virtual reality format" is an information format that uses computer-generated 3D visuals and sound to simulate real-world experiences.

[0263] "User's visual device" refers to a device that a user wears or uses to receive visual digital information, and includes, for example, smart glasses and head-mounted displays.

[0264] "Advertising information" refers to notices and promotional information presented to users for commercial or public interest purposes.

[0265] An "immersive experience" is a form of experience in which users can gain a strong sense of presence, as if they were actually in the real world.

[0266] "Personal interest-based advertising" is a method of selecting and presenting advertisements that are highly relevant to a person based on their past history and attribute information.

[0267] The system implementing this invention is server-centric and features the function of converting news information acquired using generative artificial intelligence into a virtual reality format. News information is collected in various formats such as text, images, and videos, and the server analyzes them. The analysis includes topic classification and summarization using natural language processing techniques. Based on the analysis results, a generative AI model generates 3D visuals and audio, which are then delivered as news content in virtual reality format.

[0268] Through the user's visual devices, such as smart glasses or a head-mounted display, the user receives the delivered content as an immersive experience. The server processes the 3D visuals and audio using virtual reality content generation software such as Unity or Unreal Engine. This allows the user to feel as if they are actually at the news event.

[0269] Furthermore, the server analyzes the user's personal attribute information and executes an algorithm that dynamically selects relevant advertising information and incorporates it into the virtual reality content. This allows for the effective presentation of advertisements tailored to the user's interests.

[0270] As a concrete example, consider a scenario where a user is watching news about an international sporting event. The server collects the latest match results and player comments, and uses generative AI to recreate match highlights and player interviews in 3D. Then, it inputs the prompt, "What prompt would you like to use to visualize the latest sports news and player interviews?" into the generative AI model, providing the user with an immersive experience. In this process, it is possible to naturally present specific product advertisements from national brands to stimulate the user's interest in sports equipment.

[0271] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0272] Step 1:

[0273] The server uses APIs to retrieve the latest news data from external news sources. It takes URLs or API endpoints of multiple news articles as input and stores text, images, and video data as output in local storage. Specifically, the server collects the necessary data from each information source and temporarily stores it in a structured format.

[0274] Step 2:

[0275] The server analyzes the content of the acquired news data using a natural language processing model to extract important topics and keywords. Using the previously saved text data of the news articles as input, it generates topics and summaries with an evaluation of the importance of each article as output. Specifically, the server executes an analysis algorithm to prioritize news topics.

[0276] Step 3:

[0277] The server uses a generative AI model to create prompts for generating virtual reality content based on the analyzed data. It takes topic information with assigned importance levels as input and generates prompts as output. Specifically, it prepares prompts in the format, "Generate 3D visuals and sound for this topic."

[0278] Step 4:

[0279] The generative AI model constructs 3D visuals and audio content based on the prompt text received from the server. It uses the prompt text as input and generates digital content in the form of virtual reality as output. Specifically, it utilizes the functions of Unity or Unreal Engine to draw scenes for providing visual and auditory experiences to users.

[0280] Step 5:

[0281] The server dynamically selects advertisements suitable for the user's interests considering the user's attribute information and embeds the advertisements into the generated virtual reality content. It uses the user profile data as input and generates content with advertisement insertion completed as output. Specifically, it analyzes the user's purchase history and browsing history to select relevant advertisements.

[0282] Step 6:

[0283] The user's visual device receives the virtual reality content from the server and provides an immersive news experience. It takes in the adapted 3D visual and audio data as input and presents visual and auditory information to the user as output. Specifically, the device processes the received data in real time and displays it to the user.

[0284] Furthermore, an emotion engine for estimating the user's emotions may be combined. That is, the specific processing unit 290 may estimate the user's emotions using the emotion identification model 59 and perform specific processing using the user's emotions.

[0285] The present invention is a system that combines a generative artificial intelligence and an emotion engine, and aims to improve the user's experience through the process of converting the acquired news data into the virtual reality format and distributing it to users. This system is composed of a plurality of main components including a server, a user terminal, and an emotion engine.

[0286] First, the server collects news data from multiple information sources. This data is collected in various forms such as text, images, and videos, and is analyzed by the server. Through this analysis, the important points of the news are extracted and optimized for conversion into virtual reality format.

[0287] Next, the server uses a generative artificial intelligence model to convert the news into virtual reality format. In this process, 3D objects and virtual environments are generated based on the content of the news to enable users to have a more immersive experience. In addition, the emotion engine on the terminal obtains emotion data from the user's expressions and voices and transmits it to the server.

[0288] The server utilizes this emotion data to provide an optimal news experience for the user. For example, when the user shows a surprised expression, the AI can provide additional information or visual effects that emphasize that emotion. Through such dynamic adjustments, the news viewing experience can be individualized and the user's interest can be continuously attracted.

[0289] Also, this system is beneficial for advertisers as well. By analyzing the user's emotions through the emotion engine and adjusting the advertising content based on that information, more targeted advertising campaigns can be carried out, such as displaying specific product advertisements when the user shows positive emotions, which can be said to enhance the effectiveness of the advertisement.

[0290] In this way, the system of the present invention realizes enriching the news experience in virtual reality while reflecting the user's emotions in real time, and also contributes to the optimization of advertising effects. As a specific example, when a user watches a live sports event on the news, by dynamically adjusting the video and audio according to the change in emotions at important moments of the game, the user can feel as if they were on the spot.

[0291] The following describes the processing flow.

[0292] Step 1:

[0293] The server retrieves the latest news data from APIs and information sources. The news data includes text, images, and video formats.

[0294] Step 2:

[0295] The server analyzes the acquired news data, extracting keywords and performing summarization. This organizes the important information necessary for conversion into a virtual reality format.

[0296] Step 3:

[0297] The server uses artificial intelligence to generate virtual reality content based on extracted information. Specifically, it creates 3D models and audio guides to build immersive content for users.

[0298] Step 4:

[0299] The device activates an emotion engine to recognize the user's emotions through their facial expressions and voice. The emotion engine acquires the user's emotion data in real time and sends it to the server.

[0300] Step 5:

[0301] The server analyzes the emotion data it receives and dynamically adjusts the virtual reality content according to the user's emotions. For example, if the user is surprised, the visual effects are enhanced.

[0302] Step 6:

[0303] The server selects and places ads. Ads are included appropriately based on emotions to avoid disrupting the user experience.

[0304] Step 7:

[0305] The server constructs the final virtual reality content and streams it to the user terminal. The user can start an immersive news experience on the terminal.

[0306] Step 8:

[0307] The user experiences the provided virtual reality content and appreciates the information and advertisements adjusted according to their emotions. When the user experience ends, the terminal sends interaction data to the server, which is used for further system improvement.

[0308] (Example 2)

[0309] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0310] In the conventional news distribution system, individualization considering the user's emotional reaction has not been sufficiently carried out, and the influence of news content on the user's emotions has not been maximally utilized. Also, the provision of information in the virtual reality format has not been effectively utilized, making it difficult for users to obtain an immersive experience. Furthermore, the provision of advertisements is only based on general user profile information, lacking targeting considering real-time emotional data.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0312] In this invention, the server includes means for converting information obtained using a generation artificial intelligence into a virtual reality format, means for optimizing the information in the virtual reality format based on the user's emotional data, and means for distributing the optimized virtual reality format information to the user terminal. Thereby, individualized news distribution reflecting the user's emotions in real time becomes possible, and a higher sense of immersion and effective advertisement provision are realized.

[0313] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new information and content in a natural way based on acquired data.

[0314] "Virtual reality" refers to a three-dimensional visual and auditory environment created using digital technology, providing users with an immersive experience.

[0315] "Emotional data" refers to data that indicates a user's emotional state, extracted from biometric information such as facial expressions and voice.

[0316] A "user terminal" refers to a computing device operated by a user, which is used to display virtual reality content and acquire emotional data.

[0317] "Advertising" refers to information or content presented to users for the purpose of promoting a specific product or service.

[0318] The system for implementing this invention mainly consists of multiple components, such as a server, a terminal, and a user.

[0319] First, the server collects news data from multiple sources. This collection includes obtaining data in text, image, and video formats via APIs. The server then uses natural language processing (NLP) techniques to analyze the news data and extract important information.

[0320] Next, using a generative artificial intelligence model, the server converts the analyzed news data into a virtual reality format. Specifically, it generates 3D objects and virtual environments corresponding to the news content, providing the user with an immersive experience. For example, in news reporting on a sports match, the generative AI model recreates a virtual stadium and realistic movements of the players. For this process, the generative AI model is input with a text-based prompt, "Recreate the key moments of this match."

[0321] Meanwhile, the user's device analyzes the user's facial expressions and voice via an emotion engine, acquiring emotion data in real time. This data is sent to a server and used to optimize the virtual reality content. The server dynamically adjusts the visual and auditory elements within the virtual environment according to the emotions the user expresses, creating a personalized news experience.

[0322] Furthermore, the server selects information provided by advertisers based on user profile information and sentiment data, and displays optimized ads to the target audience. As a result, users view more relevant ads, improving advertising effectiveness.

[0323] In this way, the invention reflects user emotions in real time, enabling personalized virtual reality news delivery.

[0324] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0325] Step 1:

[0326] The server collects news data. It retrieves data in text, image, and video formats from news sources via APIs. At this stage, the input is raw data provided by each source, and the output is an integrated news dataset. Within the server, this data undergoes basic data organization processes, such as being sorted by date and category.

[0327] Step 2:

[0328] The server analyzes the collected data. It uses natural language processing (NLP) techniques to extract keywords from text data and sentiment analysis to determine the tone of the news. Image and video data are analyzed using image recognition algorithms to detect visually significant points. The input to this process is integrated news data, and the output is an analyzed dataset. Specific operations include saving the information to a database.

[0329] Step 3:

[0330] The server generates virtual reality content using a generative AI model. Based on the analyzed data, it takes the prompt "Recreate this news content in a 3D environment" as input and generates a virtual environment and 3D objects. The input for this process is the analyzed news data, and the output is digital content in virtual reality format.

[0331] Step 4:

[0332] The device collects user emotion data. Using the device's camera and microphone, it captures the user's facial expressions and voice in real time, which are then analyzed by an emotion engine. The input for this step is the user's real-time biometric information, and the output is the analyzed emotion data. This information is then transferred from the device to the server.

[0333] Step 5:

[0334] The server optimizes the virtual reality content based on the acquired emotion data. The visual effects and sound elements of the content are adjusted according to the user's emotional state. The input for this step is the user's emotion data, and the output is the optimized virtual reality content. Specifically, if the user is surprised, actions such as emphasizing color effects are performed.

[0335] (Application Example 2)

[0336] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0337] In modern society, simply providing visual information is insufficient for users to have a fully immersive experience. Furthermore, the lack of information delivery tailored to users' emotions and states, as well as the optimization of advertising, means that valuable information experiences cannot be provided to users.

[0338] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0339] In this invention, the server includes means for acquiring and analyzing user emotional data, means for converting the acquired information into a virtual reality format using generative artificial intelligence, and means for dynamically adjusting the information in the virtual reality format based on the emotional data. This makes it possible to provide an immersive information experience that is tailored to the user's emotions and preferences, thereby improving the targeting effectiveness of advertisements.

[0340] "User emotion data" refers to information indicating the emotional state of a user, obtained from their facial expressions, voice, and actions.

[0341] "Means of analysis" refers to algorithms and processes used to evaluate a user's current emotional state based on acquired emotional data.

[0342] "Generative artificial intelligence" is an AI technology used for generating virtual reality content, possessing the ability to create new content based on input data.

[0343] "Means of converting to virtual reality format" refers to the technologies and processes used to construct acquired information as 3D objects or virtual environments, thereby providing users with an immersive experience.

[0344] "Means of dynamic adjustment" refers to technologies or algorithms for changing the content and display method in real time according to the user's emotional state.

[0345] "Means of inserting advertisements" refers to methods for effectively placing advertisements related to virtual reality content and presenting information without disrupting the user experience.

[0346] The system implementing this invention consists of a server, a user terminal, and an emotion engine. The specific implementation method is described below.

[0347] The server first acquires emotional data through the user's camera and microphone, and then uses an emotion recognition API to analyze it. The emotion recognition API analyzes the user's facial expressions and voice in real time and quantifies the user's emotional state. This data is used as foundational data to personalize the user's experience.

[0348] Next, the server collects news data from various sources and converts it into a virtual reality format using a generative AI model. The generative AI model generates 3D objects and virtual environments from the collected text, images, and videos, allowing users to enjoy a visually rich and immersive experience. This process utilizes virtual reality content generation software such as Unity and Unreal Engine.

[0349] Furthermore, the user's device dynamically adjusts the virtual reality content generated by the server based on the analyzed emotional data. For example, if the user expresses positive emotions, the brightness and color tone of the video can be adjusted, or the background music can be changed to improve the user experience.

[0350] Regarding advertising, targeting is performed based on user profile information and sentiment data. This allows for the effective insertion of advertisements tailored to the user's state into the virtual reality space, maximizing the effectiveness of the advertising.

[0351] For example, when a user is watching virtual reality sports news, the generative AI model can detect the user's surprise at key moments and add special effects to highlight the footage at that time. This allows the user to feel as if they are actually there.

[0352] An example of a prompt to input into a generative AI model is: "Generate the VR scene that best suits the user's emotion based on the following news event. User reaction: surprise. Event: Analysis of speeches at an important political meeting." By using this prompt, the system can provide a personalized and immersive experience for the user.

[0353] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0354] Step 1:

[0355] The server acquires emotional data through the user's device's camera and microphone. The user's facial expressions and voice are used as input and passed to an emotion recognition API. The API analyzes this data and outputs numerical data indicating the user's emotional state. This process collects the user's current feelings and reactions as digital information.

[0356] Step 2:

[0357] The server collects news data from multiple sources. Inputs include online news feeds, relevant text, images, and videos. The server processes this raw data, extracts key news points, and prepares it for a generative AI model. This prioritizes important information and filters out other information.

[0358] Step 3:

[0359] The server uses a generative AI model to convert acquired news data into a virtual reality format. The input consists of news text and image data, which the model uses to generate 3D objects and virtual environments, creating visually meaningful content for the user. The output is VR content designed to provide an immersive experience.

[0360] Step 4:

[0361] The server dynamically adjusts the generated virtual reality content based on the analyzed emotional data. Emotional state data is used as input, and the color tone and sound effects of the video are adjusted accordingly. This creates an experience that matches the user's emotions.

[0362] Step 5:

[0363] The server combines user sentiment data and profile information to select the most suitable advertisements and insert them into the virtual reality content. The input includes user profile information and sentiment data, which the AI ​​uses to select the most appropriate advertisements and seamlessly integrate them into the content. This maximizes the effectiveness of the advertisements and creates content that captures the user's attention.

[0364] Step 6:

[0365] The user terminal displays virtual reality content delivered from the server. The terminal utilizes a VR headset or display to provide the generated content to the user visually and aurally, creating an immersive news experience. During this process, dynamically adjusted video and audio are delivered to the user, enabling a more realistic virtual reality experience.

[0366] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0367] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0368] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0369] [Third Embodiment]

[0370] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0371] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0372] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0373] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0374] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0375] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0376] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0377] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0378] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0379] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0380] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0381] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0382] The present invention aims to convert news data into a virtual reality format using generative artificial intelligence and deliver it to user terminals. To implement this system, multiple servers work in cooperation to provide users with an immersive news experience.

[0383] First, the server retrieves the latest news data through APIs and other online sources. This raw news data is collected in various forms, including text, images, and videos. The server then analyzes this data and prioritizes the information based on its importance and relevance. This analysis includes classifying and summarizing news topics.

[0384] Next, the server uses generative artificial intelligence to convert the analyzed data into a virtual reality format. In this process, the AI ​​model generates 3D visuals and audio, visually and aurally representing the news event. When the user experiences this through a VR headset, they get a sense of being right there at the news event.

[0385] Furthermore, this system also has the ability to effectively insert advertisements. Based on the user's profile information and behavioral history, the server selects targeted advertisements and places them in appropriate locations within the generated virtual reality content. This process allows meaningful advertisements to be displayed without disrupting the user experience.

[0386] For example, consider a scenario where a user is watching news about an international sporting event. The server collects data including the latest match results and player comments, and uses AI to recreate match highlights and player interviews in VR format. This allows the user to experience the atmosphere of being at the venue, and the natural display of sports equipment advertisements can further stimulate interest in related products.

[0387] Thus, the system of the present invention is a new form that not only innovates conventional news distribution methods and provides users with an immersive experience, but also offers advertisers an effective promotional tool.

[0388] The following describes the processing flow.

[0389] Step 1:

[0390] The server retrieves news data in real time from APIs and other sources. The news data is collected in text, image, and video formats.

[0391] Step 2:

[0392] The server analyzes acquired news data, classifying and summarizing the information. It identifies important topics and relevant media, and converts them into a format that can be efficiently processed by generative AI models.

[0393] Step 3:

[0394] The server uses generated artificial intelligence to convert analyzed news data into a virtual reality format. This includes generating 3D models and creating stereophonic audio, resulting in a more realistic experience.

[0395] Step 4:

[0396] The server references the user's profile information and selects highly relevant advertisements. By seamlessly integrating targeted ads with virtual reality content, a visually natural ad display is achieved.

[0397] Step 5:

[0398] The server streams integrated VR content to the user's device. The user can then experience the received virtual reality content through a VR headset.

[0399] Step 6:

[0400] Users experience news through sight and sound within a VR environment. They can explore the news from a three-dimensional perspective and interact with interactive elements.

[0401] Step 7:

[0402] The device collects user interaction data and feedback. This data is sent to a server and used to improve the system and optimize advertising in the future.

[0403] (Example 1)

[0404] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0405] In today's world, a vast amount of information is generated daily, and the challenge lies in providing it to users in a meaningful way. News content, in particular, needs to be delivered quickly and in an immersive format, but traditional methods result in a limited user experience, and challenges remain in appropriately prioritizing information and effectively inserting relevant advertisements.

[0406] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0407] In this invention, the server includes means for acquiring information from multiple information sources using an information processing device, means for analyzing the acquired information and classifying and prioritizing the information based on its importance and relevance, and means for converting the analyzed information into a virtual reality format using generative artificial intelligence. This makes it possible to provide users with a more immersive news experience while appropriately inserting relevant advertisements.

[0408] An "information processing device" is a mechanical or electronic device used for collecting, processing, and transmitting data.

[0409] "Information sources" refer to sources or platforms for providing data and knowledge, including online news sites and other digital content providers.

[0410] "Generative artificial intelligence" refers to an advanced computer program or system that can automatically generate or transform data using machine learning techniques.

[0411] "Virtual reality" refers to a digital representation format that includes three-dimensional visual and auditory elements generated by a computer, providing users with an immersive experience.

[0412] "User equipment" refers to devices used by end users to receive and experience information, and includes computers and VR headsets.

[0413] "Characteristic information" refers to data related to the attributes and behavioral history of individual users, including the user's age, gender, interests, etc.

[0414] In this embodiment of the invention, a server acquires news data from an information source using an information processing device. The server can acquire news data in various formats (text, images, videos, etc.) by utilizing APIs and web crawlers connected to the internet. The acquired data is classified and prioritized through an analysis process within the server. This analysis uses natural language processing techniques and machine learning algorithms, specifically generative AI models. As a generative AI model, for example, an existing large-scale language model can be customized and used.

[0415] The server inputs the analyzed data into a generating artificial intelligence using prompts, and then generates virtual reality content. In this process, the AI ​​model generates 3D visual and audio content and converts it into a format suitable for playback on the user's device. Specific software used includes 3D content generation platforms such as Unity and Unreal Engine.

[0416] Users can experience this virtual reality content through a VR headset or compatible display device. This device receives the VR content transmitted from the server and displays it to the user in real time. Furthermore, the server refers to the user's characteristic information and selects the most suitable advertisements to integrate into the virtual reality content. This ensures that advertisements are displayed effectively without disrupting the user experience.

[0417] A concrete example is a scenario where a user experiences news about an international sporting event in VR. In this case, the server retrieves the latest match results and uses an AI model to generate match highlights in VR format. The generated content is delivered to the user's VR headset, and advertisements for related sports equipment are displayed simultaneously. An example of a prompt that enables such an experience would be, "Generate highlights of the latest international soccer match in 3D VR format and provide audio for player interviews."

[0418] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0419] Step 1:

[0420] The server collects news data from information sources. Specifically, the server retrieves news from the internet using APIs and web crawlers. The input to the server at this time is access information such as URIs and API keys necessary for retrieving the news data. The output is raw news data in various formats, including text, images, and videos.

[0421] Step 2:

[0422] The server analyzes the collected news data. In this process, the server uses natural language processing techniques to analyze the data and evaluate the importance and relevance of the news. The input data is the collected raw news information. The output includes news topic classifications, summarized information, and their priority rankings.

[0423] Step 3:

[0424] The server generates virtual reality content using artificial intelligence based on the analyzed news data. The server formats the analysis results into prompt sentences and inputs them into the AI ​​model, giving specific instructions such as "Please represent this news in 3D VR format." The input consists of the analyzed data and prompt sentences, and the output is VR content including 3D visuals and audio.

[0425] Step 4:

[0426] The server delivers the generated virtual reality content to the user. The server streams the content data to the user's terminal via the network. The input is the generated VR content, and the output is the data sent to the user's terminal.

[0427] Step 5:

[0428] The terminal receives VR content transmitted from the server and provides it to the user. Specifically, the user wears a VR headset to access a virtual reality space and experience an immersive experience. The input is content data transmitted from the server, and the output is the visual and auditory VR experience provided to the user.

[0429] Step 6:

[0430] The server selects appropriate advertisements based on user characteristics and integrates them into the VR content. The input is the user's profile data and the generated VR content, while the output is the VR content with the advertisements inserted. This process ensures that targeted advertisements are delivered effectively without compromising the user experience.

[0431] (Application Example 1)

[0432] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0433] Traditional news distribution methods only provide information in the form of text, images, and videos, limiting the sense of immersion and presence for users. Furthermore, the advertisements delivered were not optimized based on user interests, resulting in a lack of consistency in the information experience. In addition, while there is a need to improve the information experience through the use of visual devices, appropriate technologies have not been sufficiently available.

[0434] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0435] In this invention, the server includes means for converting information acquired using generative artificial intelligence into a virtual reality format, means for delivering the information converted into the virtual reality format to the user's visual device, and means for dynamically selecting and placing advertising information associated with the information in the virtual reality format. This enables the user to receive news as an immersive experience and to be presented with advertisements based on their personal interests.

[0436] "Generative artificial intelligence" is a technology that uses machine learning models to generate new information from diverse data.

[0437] "Virtual reality format" is an information format that uses computer-generated 3D visuals and sound to simulate real-world experiences.

[0438] "User's visual device" refers to a device that a user wears or uses to receive visual digital information, and includes, for example, smart glasses and head-mounted displays.

[0439] "Advertising information" refers to notices and promotional information presented to users for commercial or public interest purposes.

[0440] An "immersive experience" is a form of experience in which users can gain a strong sense of presence, as if they were actually in the real world.

[0441] "Personal interest-based advertising" is a method of selecting and presenting advertisements that are highly relevant to a person based on their past history and attribute information.

[0442] The system implementing this invention is server-centric and features the function of converting news information acquired using generative artificial intelligence into a virtual reality format. News information is collected in various formats such as text, images, and videos, and the server analyzes them. The analysis includes topic classification and summarization using natural language processing techniques. Based on the analysis results, a generative AI model generates 3D visuals and audio, which are then delivered as news content in virtual reality format.

[0443] Through the user's visual devices, such as smart glasses or a head-mounted display, the user receives the delivered content as an immersive experience. The server processes the 3D visuals and audio using virtual reality content generation software such as Unity or Unreal Engine. This allows the user to feel as if they are actually at the news event.

[0444] Furthermore, the server analyzes the user's personal attribute information and executes an algorithm that dynamically selects relevant advertising information and incorporates it into the virtual reality content. This allows for the effective presentation of advertisements tailored to the user's interests.

[0445] As a concrete example, consider a scenario where a user is watching news about an international sporting event. The server collects the latest match results and player comments, and uses generative AI to recreate match highlights and player interviews in 3D. Then, it inputs the prompt, "What prompt would you like to use to visualize the latest sports news and player interviews?" into the generative AI model, providing the user with an immersive experience. In this process, it is possible to naturally present specific product advertisements from national brands to stimulate the user's interest in sports equipment.

[0446] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0447] Step 1:

[0448] The server uses APIs to retrieve the latest news data from external news sources. It takes URLs or API endpoints of multiple news articles as input and stores text, images, and video data as output in local storage. Specifically, the server collects the necessary data from each information source and temporarily stores it in a structured format.

[0449] Step 2:

[0450] The server analyzes the content of the acquired news data using a natural language processing model to extract important topics and keywords. Using the previously saved text data of the news articles as input, it generates topics and summaries with an evaluation of the importance of each article as output. Specifically, the server executes an analysis algorithm to prioritize news topics.

[0451] Step 3:

[0452] The server uses a generative AI model to create prompts for generating virtual reality content based on the analyzed data. It takes topic information with assigned importance levels as input and generates prompts as output. Specifically, it prepares prompts in the format, "Generate 3D visuals and sound for this topic."

[0453] Step 4:

[0454] The generative AI model constructs 3D visuals and audio content based on prompt messages received from the server. It uses prompt messages as input and generates virtual reality-style digital content as output. Specifically, it utilizes the capabilities of Unity or Unreal Engine to render scenes that provide users with a visual and auditory experience.

[0455] Step 5:

[0456] The server dynamically selects advertisements that match the user's interests, taking into account the user's attribute information, and embeds them in the generated virtual reality content. It uses user profile data as input and generates content with embedded advertisements as output. Specifically, it analyzes the user's purchase and browsing history to select relevant advertisements.

[0457] Step 6:

[0458] The user's visual device receives virtual reality content from a server, providing an immersive news experience. It takes modified 3D visual and audio data as input and presents visual and auditory information to the user as output. Specifically, the device processes the received data in real time and displays it to the user.

[0459] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0460] This invention is a system that combines generative artificial intelligence and an emotion engine, aiming to improve the user experience through a process of converting acquired news data into a virtual reality format and delivering it to the user. The system consists of several main components, including a server, a user terminal, and an emotion engine.

[0461] First, the server collects news data from multiple sources. This data is collected in various formats, such as text, images, and videos, and then analyzed by the server. Through this analysis, key points of the news are extracted and optimized for conversion into a virtual reality format.

[0462] Next, the server uses a generative artificial intelligence model to convert the news into a virtual reality format. In this process, 3D objects and virtual environments are generated based on the content of the news to enhance user immersion. In addition, an emotion engine acquires emotion data from the user's facial expressions and voice on the device and sends it to the server.

[0463] The server leverages this sentiment data to provide users with the optimal news experience. For example, if a user shows a surprised expression, the AI ​​can provide additional information or visual effects that emphasize that emotion. This dynamic adjustment personalizes the news reading experience and keeps users engaged.

[0464] Furthermore, this system is also beneficial for advertisers. The emotion engine analyzes users' emotions and adjusts ad content based on that information, enabling more targeted advertising. For example, displaying ads for specific products when users are showing positive emotions can increase the effectiveness of the ads.

[0465] Thus, the system of the present invention enriches the news experience in virtual reality while reflecting the user's emotions in real time, and also contributes to optimizing advertising effectiveness. As a specific example, when a user watches a live sports event on the news, the video and sound are dynamically adjusted in response to changes in emotions at important moments in the game, allowing the user to experience a feeling as if they were actually there.

[0466] The following describes the processing flow.

[0467] Step 1:

[0468] The server retrieves the latest news data from APIs and information sources. The news data includes text, images, and video formats.

[0469] Step 2:

[0470] The server analyzes the acquired news data, extracting keywords and performing summarization. This organizes the important information necessary for conversion into a virtual reality format.

[0471] Step 3:

[0472] The server uses artificial intelligence to generate virtual reality content based on extracted information. Specifically, it creates 3D models and audio guides to build immersive content for users.

[0473] Step 4:

[0474] The device activates an emotion engine to recognize the user's emotions through their facial expressions and voice. The emotion engine acquires the user's emotion data in real time and sends it to the server.

[0475] Step 5:

[0476] The server analyzes the emotion data it receives and dynamically adjusts the virtual reality content according to the user's emotions. For example, if the user is surprised, the visual effects are enhanced.

[0477] Step 6:

[0478] The server selects and places ads. Ads are included appropriately based on emotions to avoid disrupting the user experience.

[0479] Step 7:

[0480] The server builds the final virtual reality content and streams it to the user's device. The user can then begin an immersive news experience on their device.

[0481] Step 8:

[0482] Users experience the provided virtual reality content and view information and advertisements tailored to their emotions. Once the user experience ends, the device sends interaction data to a server, which is used for further system improvements.

[0483] (Example 2)

[0484] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0485] Traditional news distribution systems lacked sufficient personalization to consider users' emotional responses, failing to maximize the impact news content had on users' emotions. Furthermore, virtual reality-based information delivery was not effectively utilized, making it difficult for users to have an immersive experience. Additionally, advertising was based solely on general user profile information, lacking real-time targeting that considered emotional data.

[0486] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0487] In this invention, the server includes means for converting information acquired using generative artificial intelligence into a virtual reality format, means for optimizing the information converted into a virtual reality format based on the user's emotional data, and means for delivering the optimized virtual reality format information to the user's terminal. This enables personalized news delivery that reflects the user's emotions in real time, resulting in a higher level of immersion and more effective advertising.

[0488] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new information and content in a natural way based on acquired data.

[0489] "Virtual reality" refers to a three-dimensional visual and auditory environment created using digital technology, providing users with an immersive experience.

[0490] "Emotional data" refers to data that indicates a user's emotional state, extracted from biometric information such as facial expressions and voice.

[0491] A "user terminal" refers to a computing device operated by a user, which is used to display virtual reality content and acquire emotional data.

[0492] "Advertising" refers to information or content presented to users for the purpose of promoting a specific product or service.

[0493] The system for implementing this invention mainly consists of multiple components, such as a server, a terminal, and a user.

[0494] First, the server collects news data from multiple sources. This collection includes obtaining data in text, image, and video formats via APIs. The server then uses natural language processing (NLP) techniques to analyze the news data and extract important information.

[0495] Next, using a generative artificial intelligence model, the server converts the analyzed news data into a virtual reality format. Specifically, it generates 3D objects and virtual environments corresponding to the news content, providing the user with an immersive experience. For example, in news reporting on a sports match, the generative AI model recreates a virtual stadium and realistic movements of the players. For this process, the generative AI model is input with a text-based prompt, "Recreate the key moments of this match."

[0496] Meanwhile, the user's device analyzes the user's facial expressions and voice via an emotion engine, acquiring emotion data in real time. This data is sent to a server and used to optimize the virtual reality content. The server dynamically adjusts the visual and auditory elements within the virtual environment according to the emotions the user expresses, creating a personalized news experience.

[0497] Furthermore, the server selects information provided by advertisers based on user profile information and sentiment data, and displays optimized ads to the target audience. As a result, users view more relevant ads, improving advertising effectiveness.

[0498] In this way, the invention reflects user emotions in real time, enabling personalized virtual reality news delivery.

[0499] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0500] Step 1:

[0501] The server collects news data. It retrieves data in text, image, and video formats from news sources via APIs. At this stage, the input is raw data provided by each source, and the output is an integrated news dataset. Within the server, this data undergoes basic data organization processes, such as being sorted by date and category.

[0502] Step 2:

[0503] The server analyzes the collected data. It uses natural language processing (NLP) techniques to extract keywords from text data and sentiment analysis to determine the tone of the news. Image and video data are analyzed using image recognition algorithms to detect visually significant points. The input to this process is integrated news data, and the output is an analyzed dataset. Specific operations include saving the information to a database.

[0504] Step 3:

[0505] The server generates virtual reality content using a generative AI model. Based on the analyzed data, it takes the prompt "Recreate this news content in a 3D environment" as input and generates a virtual environment and 3D objects. The input for this process is the analyzed news data, and the output is digital content in virtual reality format.

[0506] Step 4:

[0507] The device collects user emotion data. Using the device's camera and microphone, it captures the user's facial expressions and voice in real time, which are then analyzed by an emotion engine. The input for this step is the user's real-time biometric information, and the output is the analyzed emotion data. This information is then transferred from the device to the server.

[0508] Step 5:

[0509] The server optimizes the virtual reality content based on the acquired emotion data. The visual effects and sound elements of the content are adjusted according to the user's emotional state. The input for this step is the user's emotion data, and the output is the optimized virtual reality content. Specifically, if the user is surprised, actions such as emphasizing color effects are performed.

[0510] (Application Example 2)

[0511] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0512] In modern society, simply providing visual information is insufficient for users to have a fully immersive experience. Furthermore, the lack of information delivery tailored to users' emotions and states, as well as the optimization of advertising, means that valuable information experiences cannot be provided to users.

[0513] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0514] In this invention, the server includes means for acquiring and analyzing user emotional data, means for converting the acquired information into a virtual reality format using generative artificial intelligence, and means for dynamically adjusting the information in the virtual reality format based on the emotional data. This makes it possible to provide an immersive information experience that is tailored to the user's emotions and preferences, thereby improving the targeting effectiveness of advertisements.

[0515] "User emotion data" refers to information indicating the emotional state of a user, obtained from their facial expressions, voice, and actions.

[0516] "Means of analysis" refers to algorithms and processes used to evaluate a user's current emotional state based on acquired emotional data.

[0517] "Generative artificial intelligence" is an AI technology used for generating virtual reality content, possessing the ability to create new content based on input data.

[0518] "Means of converting to virtual reality format" refers to the technologies and processes used to construct acquired information as 3D objects or virtual environments, thereby providing users with an immersive experience.

[0519] "Means of dynamic adjustment" refers to technologies or algorithms for changing the content and display method in real time according to the user's emotional state.

[0520] "Means of inserting advertisements" refers to methods for effectively placing advertisements related to virtual reality content and presenting information without disrupting the user experience.

[0521] The system implementing this invention consists of a server, a user terminal, and an emotion engine. The specific implementation method is described below.

[0522] The server first acquires emotional data through the user's camera and microphone, and then uses an emotion recognition API to analyze it. The emotion recognition API analyzes the user's facial expressions and voice in real time and quantifies the user's emotional state. This data is used as foundational data to personalize the user's experience.

[0523] Next, the server collects news data from various sources and converts it into a virtual reality format using a generative AI model. The generative AI model generates 3D objects and virtual environments from the collected text, images, and videos, allowing users to enjoy a visually rich and immersive experience. This process utilizes virtual reality content generation software such as Unity and Unreal Engine.

[0524] Furthermore, the user's device dynamically adjusts the virtual reality content generated by the server based on the analyzed emotional data. For example, if the user expresses positive emotions, the brightness and color tone of the video can be adjusted, or the background music can be changed to improve the user experience.

[0525] Regarding advertising, targeting is performed based on user profile information and sentiment data. This allows for the effective insertion of advertisements tailored to the user's state into the virtual reality space, maximizing the effectiveness of the advertising.

[0526] For example, when a user is watching virtual reality sports news, the generative AI model can detect the user's surprise at key moments and add special effects to highlight the footage at that time. This allows the user to feel as if they are actually there.

[0527] An example of a prompt to input into a generative AI model is: "Generate the VR scene that best suits the user's emotion based on the following news event. User reaction: surprise. Event: Analysis of speeches at an important political meeting." By using this prompt, the system can provide a personalized and immersive experience for the user.

[0528] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0529] Step 1:

[0530] The server acquires emotional data through the user's device's camera and microphone. The user's facial expressions and voice are used as input and passed to an emotion recognition API. The API analyzes this data and outputs numerical data indicating the user's emotional state. This process collects the user's current feelings and reactions as digital information.

[0531] Step 2:

[0532] The server collects news data from multiple sources. Inputs include online news feeds, relevant text, images, and videos. The server processes this raw data, extracts key news points, and prepares it for a generative AI model. This prioritizes important information and filters out other information.

[0533] Step 3:

[0534] The server uses a generative AI model to convert acquired news data into a virtual reality format. The input consists of news text and image data, which the model uses to generate 3D objects and virtual environments, creating visually meaningful content for the user. The output is VR content designed to provide an immersive experience.

[0535] Step 4:

[0536] The server dynamically adjusts the generated virtual reality content based on the analyzed emotional data. Emotional state data is used as input, and the color tone and sound effects of the video are adjusted accordingly. This creates an experience that matches the user's emotions.

[0537] Step 5:

[0538] The server combines user sentiment data and profile information to select the most suitable advertisements and insert them into the virtual reality content. The input includes user profile information and sentiment data, which the AI ​​uses to select the most appropriate advertisements and seamlessly integrate them into the content. This maximizes the effectiveness of the advertisements and creates content that captures the user's attention.

[0539] Step 6:

[0540] The user terminal displays virtual reality content delivered from the server. The terminal utilizes a VR headset or display to provide the generated content to the user visually and aurally, creating an immersive news experience. During this process, dynamically adjusted video and audio are delivered to the user, enabling a more realistic virtual reality experience.

[0541] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0542] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0543] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0544] [Fourth Embodiment]

[0545] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0546] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0547] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0548] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0549] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0550] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0551] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0552] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0553] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0554] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0555] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0556] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0557] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0558] The present invention aims to convert news data into a virtual reality format using generative artificial intelligence and deliver it to user terminals. To implement this system, multiple servers work in cooperation to provide users with an immersive news experience.

[0559] First, the server retrieves the latest news data through APIs and other online sources. This raw news data is collected in various forms, including text, images, and videos. The server then analyzes this data and prioritizes the information based on its importance and relevance. This analysis includes classifying and summarizing news topics.

[0560] Next, the server uses generative artificial intelligence to convert the analyzed data into a virtual reality format. In this process, the AI ​​model generates 3D visuals and audio, visually and aurally representing the news event. When the user experiences this through a VR headset, they get a sense of being right there at the news event.

[0561] Furthermore, this system also has the ability to effectively insert advertisements. Based on the user's profile information and behavioral history, the server selects targeted advertisements and places them in appropriate locations within the generated virtual reality content. This process allows meaningful advertisements to be displayed without disrupting the user experience.

[0562] For example, consider a scenario where a user is watching news about an international sporting event. The server collects data including the latest match results and player comments, and uses AI to recreate match highlights and player interviews in VR format. This allows the user to experience the atmosphere of being at the venue, and the natural display of sports equipment advertisements can further stimulate interest in related products.

[0563] Thus, the system of the present invention is a new form that not only innovates conventional news distribution methods and provides users with an immersive experience, but also offers advertisers an effective promotional tool.

[0564] The following describes the processing flow.

[0565] Step 1:

[0566] The server retrieves news data in real time from APIs and other sources. The news data is collected in text, image, and video formats.

[0567] Step 2:

[0568] The server analyzes acquired news data, classifying and summarizing the information. It identifies important topics and relevant media, and converts them into a format that can be efficiently processed by generative AI models.

[0569] Step 3:

[0570] The server uses generated artificial intelligence to convert analyzed news data into a virtual reality format. This includes generating 3D models and creating stereophonic audio, resulting in a more realistic experience.

[0571] Step 4:

[0572] The server references the user's profile information and selects highly relevant advertisements. By seamlessly integrating targeted ads with virtual reality content, a visually natural ad display is achieved.

[0573] Step 5:

[0574] The server streams integrated VR content to the user's device. The user can then experience the received virtual reality content through a VR headset.

[0575] Step 6:

[0576] Users experience news through sight and sound within a VR environment. They can explore the news from a three-dimensional perspective and interact with interactive elements.

[0577] Step 7:

[0578] The device collects user interaction data and feedback. This data is sent to a server and used to improve the system and optimize advertising in the future.

[0579] (Example 1)

[0580] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0581] In today's world, a vast amount of information is generated daily, and the challenge lies in providing it to users in a meaningful way. News content, in particular, needs to be delivered quickly and in an immersive format, but traditional methods result in a limited user experience, and challenges remain in appropriately prioritizing information and effectively inserting relevant advertisements.

[0582] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0583] In this invention, the server includes means for acquiring information from multiple information sources using an information processing device, means for analyzing the acquired information and classifying and prioritizing the information based on its importance and relevance, and means for converting the analyzed information into a virtual reality format using generative artificial intelligence. This makes it possible to provide users with a more immersive news experience while appropriately inserting relevant advertisements.

[0584] An "information processing device" is a mechanical or electronic device used for collecting, processing, and transmitting data.

[0585] "Information sources" refer to sources or platforms for providing data and knowledge, including online news sites and other digital content providers.

[0586] "Generative artificial intelligence" refers to an advanced computer program or system that can automatically generate or transform data using machine learning techniques.

[0587] "Virtual reality" refers to a digital representation format that includes three-dimensional visual and auditory elements generated by a computer, providing users with an immersive experience.

[0588] "User equipment" refers to devices used by end users to receive and experience information, and includes computers and VR headsets.

[0589] "Characteristic information" refers to data related to the attributes and behavioral history of individual users, including the user's age, gender, interests, etc.

[0590] In this embodiment of the invention, a server acquires news data from an information source using an information processing device. The server can acquire news data in various formats (text, images, videos, etc.) by utilizing APIs and web crawlers connected to the internet. The acquired data is classified and prioritized through an analysis process within the server. This analysis uses natural language processing techniques and machine learning algorithms, specifically generative AI models. As a generative AI model, for example, an existing large-scale language model can be customized and used.

[0591] The server inputs the analyzed data into a generating artificial intelligence using prompts, and then generates virtual reality content. In this process, the AI ​​model generates 3D visual and audio content and converts it into a format suitable for playback on the user's device. Specific software used includes 3D content generation platforms such as Unity and Unreal Engine.

[0592] Users can experience this virtual reality content through a VR headset or compatible display device. This device receives the VR content transmitted from the server and displays it to the user in real time. Furthermore, the server refers to the user's characteristic information and selects the most suitable advertisements to integrate into the virtual reality content. This ensures that advertisements are displayed effectively without disrupting the user experience.

[0593] A concrete example is a scenario where a user experiences news about an international sporting event in VR. In this case, the server retrieves the latest match results and uses an AI model to generate match highlights in VR format. The generated content is delivered to the user's VR headset, and advertisements for related sports equipment are displayed simultaneously. An example of a prompt that enables such an experience would be, "Generate highlights of the latest international soccer match in 3D VR format and provide audio for player interviews."

[0594] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0595] Step 1:

[0596] The server collects news data from information sources. Specifically, the server retrieves news from the internet using APIs and web crawlers. The input to the server at this time is access information such as URIs and API keys necessary for retrieving the news data. The output is raw news data in various formats, including text, images, and videos.

[0597] Step 2:

[0598] The server analyzes the collected news data. In this process, the server uses natural language processing techniques to analyze the data and evaluate the importance and relevance of the news. The input data is the collected raw news information. The output includes news topic classifications, summarized information, and their priority rankings.

[0599] Step 3:

[0600] The server generates virtual reality content using artificial intelligence based on the analyzed news data. The server formats the analysis results into prompt sentences and inputs them into the AI ​​model, giving specific instructions such as "Please represent this news in 3D VR format." The input consists of the analyzed data and prompt sentences, and the output is VR content including 3D visuals and audio.

[0601] Step 4:

[0602] The server delivers the generated virtual reality content to the user. The server streams the content data to the user's terminal via the network. The input is the generated VR content, and the output is the data sent to the user's terminal.

[0603] Step 5:

[0604] The terminal receives VR content transmitted from the server and provides it to the user. Specifically, the user wears a VR headset to access a virtual reality space and experience an immersive experience. The input is content data transmitted from the server, and the output is the visual and auditory VR experience provided to the user.

[0605] Step 6:

[0606] The server selects appropriate advertisements based on user characteristics and integrates them into the VR content. The input is the user's profile data and the generated VR content, while the output is the VR content with the advertisements inserted. This process ensures that targeted advertisements are delivered effectively without compromising the user experience.

[0607] (Application Example 1)

[0608] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0609] Traditional news distribution methods only provide information in the form of text, images, and videos, limiting the sense of immersion and presence for users. Furthermore, the advertisements delivered were not optimized based on user interests, resulting in a lack of consistency in the information experience. In addition, while there is a need to improve the information experience through the use of visual devices, appropriate technologies have not been sufficiently available.

[0610] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0611] In this invention, the server includes means for converting information acquired using generative artificial intelligence into a virtual reality format, means for delivering the information converted into the virtual reality format to the user's visual device, and means for dynamically selecting and placing advertising information associated with the information in the virtual reality format. This enables the user to receive news as an immersive experience and to be presented with advertisements based on their personal interests.

[0612] "Generative artificial intelligence" is a technology that uses machine learning models to generate new information from diverse data.

[0613] "Virtual reality format" is an information format that uses computer-generated 3D visuals and sound to simulate real-world experiences.

[0614] "User's visual device" refers to a device that a user wears or uses to receive visual digital information, and includes, for example, smart glasses and head-mounted displays.

[0615] "Advertising information" refers to notices and promotional information presented to users for commercial or public interest purposes.

[0616] An "immersive experience" is a form of experience in which users can gain a strong sense of presence, as if they were actually in the real world.

[0617] "Personal interest-based advertising" is a method of selecting and presenting advertisements that are highly relevant to a person based on their past history and attribute information.

[0618] The system implementing this invention is server-centric and features the function of converting news information acquired using generative artificial intelligence into a virtual reality format. News information is collected in various formats such as text, images, and videos, and the server analyzes them. The analysis includes topic classification and summarization using natural language processing techniques. Based on the analysis results, a generative AI model generates 3D visuals and audio, which are then delivered as news content in virtual reality format.

[0619] Through the user's visual devices, such as smart glasses or a head-mounted display, the user receives the delivered content as an immersive experience. The server processes the 3D visuals and audio using virtual reality content generation software such as Unity or Unreal Engine. This allows the user to feel as if they are actually at the news event.

[0620] Furthermore, the server analyzes the user's personal attribute information and executes an algorithm that dynamically selects relevant advertising information and incorporates it into the virtual reality content. This allows for the effective presentation of advertisements tailored to the user's interests.

[0621] As a concrete example, consider a scenario where a user is watching news about an international sporting event. The server collects the latest match results and player comments, and uses generative AI to recreate match highlights and player interviews in 3D. Then, it inputs the prompt, "What prompt would you like to use to visualize the latest sports news and player interviews?" into the generative AI model, providing the user with an immersive experience. In this process, it is possible to naturally present specific product advertisements from national brands to stimulate the user's interest in sports equipment.

[0622] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0623] Step 1:

[0624] The server uses APIs to retrieve the latest news data from external news sources. It takes URLs or API endpoints of multiple news articles as input and stores text, images, and video data as output in local storage. Specifically, the server collects the necessary data from each information source and temporarily stores it in a structured format.

[0625] Step 2:

[0626] The server analyzes the content of the acquired news data using a natural language processing model to extract important topics and keywords. Using the previously saved text data of the news articles as input, it generates topics and summaries with an evaluation of the importance of each article as output. Specifically, the server executes an analysis algorithm to prioritize news topics.

[0627] Step 3:

[0628] The server uses a generative AI model to create prompts for generating virtual reality content based on the analyzed data. It takes topic information with assigned importance levels as input and generates prompts as output. Specifically, it prepares prompts in the format, "Generate 3D visuals and sound for this topic."

[0629] Step 4:

[0630] The generative AI model constructs 3D visuals and audio content based on prompt messages received from the server. It uses prompt messages as input and generates virtual reality-style digital content as output. Specifically, it utilizes the capabilities of Unity or Unreal Engine to render scenes that provide users with a visual and auditory experience.

[0631] Step 5:

[0632] The server dynamically selects advertisements that match the user's interests, taking into account the user's attribute information, and embeds them in the generated virtual reality content. It uses user profile data as input and generates content with embedded advertisements as output. Specifically, it analyzes the user's purchase and browsing history to select relevant advertisements.

[0633] Step 6:

[0634] The user's visual device receives virtual reality content from a server, providing an immersive news experience. It takes modified 3D visual and audio data as input and presents visual and auditory information to the user as output. Specifically, the device processes the received data in real time and displays it to the user.

[0635] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0636] This invention is a system that combines generative artificial intelligence and an emotion engine, aiming to improve the user experience through a process of converting acquired news data into a virtual reality format and delivering it to the user. The system consists of several main components, including a server, a user terminal, and an emotion engine.

[0637] First, the server collects news data from multiple sources. This data is collected in various formats, such as text, images, and videos, and then analyzed by the server. Through this analysis, key points of the news are extracted and optimized for conversion into a virtual reality format.

[0638] Next, the server uses a generative artificial intelligence model to convert the news into a virtual reality format. In this process, 3D objects and virtual environments are generated based on the content of the news to enhance user immersion. In addition, an emotion engine acquires emotion data from the user's facial expressions and voice on the device and sends it to the server.

[0639] The server leverages this sentiment data to provide users with the optimal news experience. For example, if a user shows a surprised expression, the AI ​​can provide additional information or visual effects that emphasize that emotion. This dynamic adjustment personalizes the news reading experience and keeps users engaged.

[0640] Furthermore, this system is also beneficial for advertisers. The emotion engine analyzes users' emotions and adjusts ad content based on that information, enabling more targeted advertising. For example, displaying ads for specific products when users are showing positive emotions can increase the effectiveness of the ads.

[0641] Thus, the system of the present invention enriches the news experience in virtual reality while reflecting the user's emotions in real time, and also contributes to optimizing advertising effectiveness. As a specific example, when a user watches a live sports event on the news, the video and sound are dynamically adjusted in response to changes in emotions at important moments in the game, allowing the user to experience a feeling as if they were actually there.

[0642] The following describes the processing flow.

[0643] Step 1:

[0644] The server retrieves the latest news data from APIs and information sources. The news data includes text, images, and video formats.

[0645] Step 2:

[0646] The server analyzes the acquired news data, extracting keywords and performing summarization. This organizes the important information necessary for conversion into a virtual reality format.

[0647] Step 3:

[0648] The server uses artificial intelligence to generate virtual reality content based on extracted information. Specifically, it creates 3D models and audio guides to build immersive content for users.

[0649] Step 4:

[0650] The device activates an emotion engine to recognize the user's emotions through their facial expressions and voice. The emotion engine acquires the user's emotion data in real time and sends it to the server.

[0651] Step 5:

[0652] The server analyzes the emotion data it receives and dynamically adjusts the virtual reality content according to the user's emotions. For example, if the user is surprised, the visual effects are enhanced.

[0653] Step 6:

[0654] The server selects and places ads. Ads are included appropriately based on emotions to avoid disrupting the user experience.

[0655] Step 7:

[0656] The server builds the final virtual reality content and streams it to the user's device. The user can then begin an immersive news experience on their device.

[0657] Step 8:

[0658] Users experience the provided virtual reality content and view information and advertisements tailored to their emotions. Once the user experience ends, the device sends interaction data to a server, which is used for further system improvements.

[0659] (Example 2)

[0660] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0661] Traditional news distribution systems lacked sufficient personalization to consider users' emotional responses, failing to maximize the impact news content had on users' emotions. Furthermore, virtual reality-based information delivery was not effectively utilized, making it difficult for users to have an immersive experience. Additionally, advertising was based solely on general user profile information, lacking real-time targeting that considered emotional data.

[0662] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0663] In this invention, the server includes means for converting information acquired using generative artificial intelligence into a virtual reality format, means for optimizing the information converted into a virtual reality format based on the user's emotional data, and means for delivering the optimized virtual reality format information to the user's terminal. This enables personalized news delivery that reflects the user's emotions in real time, resulting in a higher level of immersion and more effective advertising.

[0664] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new information and content in a natural way based on acquired data.

[0665] "Virtual reality" refers to a three-dimensional visual and auditory environment created using digital technology, providing users with an immersive experience.

[0666] "Emotional data" refers to data that indicates a user's emotional state, extracted from biometric information such as facial expressions and voice.

[0667] A "user terminal" refers to a computing device operated by a user, which is used to display virtual reality content and acquire emotional data.

[0668] "Advertising" refers to information or content presented to users for the purpose of promoting a specific product or service.

[0669] The system for implementing this invention mainly consists of multiple components, such as a server, a terminal, and a user.

[0670] First, the server collects news data from multiple sources. This collection includes obtaining data in text, image, and video formats via APIs. The server then uses natural language processing (NLP) techniques to analyze the news data and extract important information.

[0671] Next, using a generative artificial intelligence model, the server converts the analyzed news data into a virtual reality format. Specifically, it generates 3D objects and virtual environments corresponding to the news content, providing the user with an immersive experience. For example, in news reporting on a sports match, the generative AI model recreates a virtual stadium and realistic movements of the players. For this process, the generative AI model is input with a text-based prompt, "Recreate the key moments of this match."

[0672] Meanwhile, the user's device analyzes the user's facial expressions and voice via an emotion engine, acquiring emotion data in real time. This data is sent to a server and used to optimize the virtual reality content. The server dynamically adjusts the visual and auditory elements within the virtual environment according to the emotions the user expresses, creating a personalized news experience.

[0673] Furthermore, the server selects information provided by advertisers based on user profile information and sentiment data, and displays optimized ads to the target audience. As a result, users view more relevant ads, improving advertising effectiveness.

[0674] In this way, the invention reflects user emotions in real time, enabling personalized virtual reality news delivery.

[0675] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0676] Step 1:

[0677] The server collects news data. It retrieves data in text, image, and video formats from news sources via APIs. At this stage, the input is raw data provided by each source, and the output is an integrated news dataset. Within the server, this data undergoes basic data organization processes, such as being sorted by date and category.

[0678] Step 2:

[0679] The server analyzes the collected data. It uses natural language processing (NLP) techniques to extract keywords from text data and sentiment analysis to determine the tone of the news. Image and video data are analyzed using image recognition algorithms to detect visually significant points. The input to this process is integrated news data, and the output is an analyzed dataset. Specific operations include saving the information to a database.

[0680] Step 3:

[0681] The server generates virtual reality content using a generative AI model. Based on the analyzed data, it takes the prompt "Recreate this news content in a 3D environment" as input and generates a virtual environment and 3D objects. The input for this process is the analyzed news data, and the output is digital content in virtual reality format.

[0682] Step 4:

[0683] The device collects user emotion data. Using the device's camera and microphone, it captures the user's facial expressions and voice in real time, which are then analyzed by an emotion engine. The input for this step is the user's real-time biometric information, and the output is the analyzed emotion data. This information is then transferred from the device to the server.

[0684] Step 5:

[0685] The server optimizes the virtual reality content based on the acquired emotion data. The visual effects and sound elements of the content are adjusted according to the user's emotional state. The input for this step is the user's emotion data, and the output is the optimized virtual reality content. Specifically, if the user is surprised, actions such as emphasizing color effects are performed.

[0686] (Application Example 2)

[0687] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0688] In modern society, simply providing visual information is insufficient for users to have a fully immersive experience. Furthermore, the lack of information delivery tailored to users' emotions and states, as well as the optimization of advertising, means that valuable information experiences cannot be provided to users.

[0689] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0690] In this invention, the server includes means for acquiring and analyzing user emotional data, means for converting the acquired information into a virtual reality format using generative artificial intelligence, and means for dynamically adjusting the information in the virtual reality format based on the emotional data. This makes it possible to provide an immersive information experience that is tailored to the user's emotions and preferences, thereby improving the targeting effectiveness of advertisements.

[0691] "User emotion data" refers to information indicating the emotional state of a user, obtained from their facial expressions, voice, and actions.

[0692] "Means of analysis" refers to algorithms and processes used to evaluate a user's current emotional state based on acquired emotional data.

[0693] "Generative artificial intelligence" is an AI technology used for generating virtual reality content, possessing the ability to create new content based on input data.

[0694] "Means of converting to virtual reality format" refers to the technologies and processes used to construct acquired information as 3D objects or virtual environments, thereby providing users with an immersive experience.

[0695] "Means of dynamic adjustment" refers to technologies or algorithms for changing the content and display method in real time according to the user's emotional state.

[0696] "Means of inserting advertisements" refers to methods for effectively placing advertisements related to virtual reality content and presenting information without disrupting the user experience.

[0697] The system implementing this invention consists of a server, a user terminal, and an emotion engine. The specific implementation method is described below.

[0698] The server first acquires emotional data through the user's camera and microphone, and then uses an emotion recognition API to analyze it. The emotion recognition API analyzes the user's facial expressions and voice in real time and quantifies the user's emotional state. This data is used as foundational data to personalize the user's experience.

[0699] Next, the server collects news data from various sources and converts it into a virtual reality format using a generative AI model. The generative AI model generates 3D objects and virtual environments from the collected text, images, and videos, allowing users to enjoy a visually rich and immersive experience. This process utilizes virtual reality content generation software such as Unity and Unreal Engine.

[0700] Furthermore, the user's device dynamically adjusts the virtual reality content generated by the server based on the analyzed emotional data. For example, if the user expresses positive emotions, the brightness and color tone of the video can be adjusted, or the background music can be changed to improve the user experience.

[0701] Regarding advertising, targeting is performed based on user profile information and sentiment data. This allows for the effective insertion of advertisements tailored to the user's state into the virtual reality space, maximizing the effectiveness of the advertising.

[0702] For example, when a user is watching virtual reality sports news, the generative AI model can detect the user's surprise at key moments and add special effects to highlight the footage at that time. This allows the user to feel as if they are actually there.

[0703] An example of a prompt to input into a generative AI model is: "Generate the VR scene that best suits the user's emotion based on the following news event. User reaction: surprise. Event: Analysis of speeches at an important political meeting." By using this prompt, the system can provide a personalized and immersive experience for the user.

[0704] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0705] Step 1:

[0706] The server acquires emotional data through the user's device's camera and microphone. The user's facial expressions and voice are used as input and passed to an emotion recognition API. The API analyzes this data and outputs numerical data indicating the user's emotional state. This process collects the user's current feelings and reactions as digital information.

[0707] Step 2:

[0708] The server collects news data from multiple sources. Inputs include online news feeds, relevant text, images, and videos. The server processes this raw data, extracts key news points, and prepares it for a generative AI model. This prioritizes important information and filters out other information.

[0709] Step 3:

[0710] The server uses a generative AI model to convert acquired news data into a virtual reality format. The input consists of news text and image data, which the model uses to generate 3D objects and virtual environments, creating visually meaningful content for the user. The output is VR content designed to provide an immersive experience.

[0711] Step 4:

[0712] The server dynamically adjusts the generated virtual reality content based on the analyzed emotional data. Emotional state data is used as input, and the color tone and sound effects of the video are adjusted accordingly. This creates an experience that matches the user's emotions.

[0713] Step 5:

[0714] The server combines user sentiment data and profile information to select the most suitable advertisements and insert them into the virtual reality content. The input includes user profile information and sentiment data, which the AI ​​uses to select the most appropriate advertisements and seamlessly integrate them into the content. This maximizes the effectiveness of the advertisements and creates content that captures the user's attention.

[0715] Step 6:

[0716] The user terminal displays virtual reality content delivered from the server. The terminal utilizes a VR headset or display to provide the generated content to the user visually and aurally, creating an immersive news experience. During this process, dynamically adjusted video and audio are delivered to the user, enabling a more realistic virtual reality experience.

[0717] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0718] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0719] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0720] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0721] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0722] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0723] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0724] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0725] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0726] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0727] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0728] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0729] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0730] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0731] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0732] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0733] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0734] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0735] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0736] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0737] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0738] The following is further disclosed regarding the embodiments described above.

[0739] (Claim 1)

[0740] A means of converting information acquired using generative artificial intelligence into a virtual reality format,

[0741] A means for distributing the information converted into the aforementioned virtual reality format to a user terminal,

[0742] Means for inserting advertisements associated with the aforementioned virtual reality format information,

[0743] A system that includes this.

[0744] (Claim 2)

[0745] The system according to claim 1, further comprising means for analyzing acquired information and extracting important information.

[0746] (Claim 3)

[0747] The system according to claim 1, further comprising means for selecting advertisements based on user profile information.

[0748] "Example 1"

[0749] (Claim 1)

[0750] A means of acquiring information from multiple information sources using an information processing device,

[0751] A means for analyzing acquired information and classifying and prioritizing it based on its importance and relevance,

[0752] A means of converting information analyzed using generative artificial intelligence into a virtual reality format,

[0753] A means for transmitting information converted into a virtual reality format to a user device,

[0754] A means for inserting advertisements associated with generated virtual reality information,

[0755] A method for selecting advertisements based on user characteristic information,

[0756] A system that includes this.

[0757] (Claim 2)

[0758] The system according to claim 1, further comprising means for generating visual and auditory representations of acquired information.

[0759] (Claim 3)

[0760] The system according to claim 1, further comprising means for streaming information in virtual reality format and playing back the information on a user-installed device.

[0761] "Application Example 1"

[0762] (Claim 1)

[0763] A means of converting information acquired using generative artificial intelligence into a virtual reality format,

[0764] Means for delivering the information converted into the aforementioned virtual reality format to the user's visual device,

[0765] Means for dynamically selecting and arranging advertising information associated with the aforementioned virtual reality format information,

[0766] A means by which a user receives news as an immersive experience using the aforementioned visual device,

[0767] A system that includes this.

[0768] (Claim 2)

[0769] The system according to claim 1, further comprising means for analyzing acquired information, extracting important information, and generating 3D visual and audio elements based thereon.

[0770] (Claim 3)

[0771] The system according to claim 1, further comprising means for selecting advertising information based on a user's personal attribute information and incorporating it into information in a virtual reality format.

[0772] "Example 2 of combining an emotion engine"

[0773] (Claim 1)

[0774] A means of converting information acquired using generative artificial intelligence into a virtual reality format,

[0775] A means for optimizing the information converted into the virtual reality format based on the user's emotional data,

[0776] A means for distributing the optimized virtual reality format information to the user terminal,

[0777] Means for inserting advertisements associated with the aforementioned virtual reality format information,

[0778] A system that includes this.

[0779] (Claim 2)

[0780] The system according to claim 1, further comprising means for analyzing acquired information and extracting important information.

[0781] (Claim 3)

[0782] The system according to claim 1, further comprising means for selecting advertisements based on user profile information and sentiment data.

[0783] "Application example 2 when combining with an emotional engine"

[0784] (Claim 1)

[0785] A means of acquiring and analyzing user sentiment data,

[0786] A means of converting information acquired using generative artificial intelligence into a virtual reality format,

[0787] A means for distributing the information converted into the aforementioned virtual reality format to a user terminal,

[0788] A means of dynamically adjusting virtual reality information based on emotional data,

[0789] Means for inserting advertisements associated with the aforementioned virtual reality format information,

[0790] A system that includes this.

[0791] (Claim 2)

[0792] The system according to claim 1, further comprising means for analyzing acquired information and extracting important information.

[0793] (Claim 3)

[0794] The system according to claim 1, further comprising means for selecting advertisements based on user profile information and sentiment data. [Explanation of Symbols]

[0795] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of converting information acquired using generative artificial intelligence into a virtual reality format, A means for distributing the information converted into the aforementioned virtual reality format to a user terminal, Means for inserting advertisements associated with the aforementioned virtual reality format information, A system that includes this.

2. The system according to claim 1, further comprising means for analyzing acquired information and extracting important information.

3. The system according to claim 1, further comprising means for selecting advertisements based on user profile information.

Images