system

Abstract

We provide the system. [Solution] A system that provides games using augmented reality technology using information terminal devices used by multiple users in a park, A means of collecting participation information from users in the park and generating game content based on that information, A means for distributing the generated game content to multiple information terminal devices, A means for dynamically generating game scenes based on the user's location information acquired via the aforementioned information terminal device, A means of converting points accumulated by users according to completed game missions into points for an electronic payment system, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance responding to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, there is a problem that face-to-face communication among neighbors and local residents is becoming scarce. In particular, the problem is that the function of a park as a place for communication that it should originally have is not fully exerted. The object of the present invention is to provide a means for users visiting a park to communicate with each other in a natural form and to promote the formation of a richer community.

Means for Solving the Problems

[0005] This invention provides a game system utilizing augmented reality technology that users can participate in using information terminal devices in a park. The system first collects participation information from users and generates appropriate game content based on that information. The generated game content is distributed to multiple information terminal devices, allowing each user to experience dynamically generated game scenes in real time. Furthermore, users are awarded points for missions completed as a result of the game, and these points are converted through an electronic payment system. This system provides opportunities for interpersonal interaction within the park and contributes to the revitalization of the local community.

[0006] An "information terminal device" is a portable electronic device that can display games and provide an interface using communication functions and augmented reality technology.

[0007] Augmented reality technology is a technology that overlays digital information onto images of the real world to provide users with a visual experience.

[0008] "Game content generation" is the process of creating distinctive game elements and missions that participants can enjoy, based on collected user information and environmental data.

[0009] An "electronic payment system" is an online transaction platform that allows for safe and rapid transactions using digitized points or currencies.

[0010] "Points" are quantifiable rewards awarded to users based on their activities and achievements within the game, and are later converted into actual value through an electronic payment system. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention relates to a specific embodiment of a game system using augmented reality technology in which users participate using information terminal devices in a park.

[0033] The server uses an AI algorithm to generate game content tailored to participants, based on park map data and current environmental information. The generated game data is delivered to participants' information terminals in real time. This game data includes the location of game objects, mission details, and methods for interaction between participants.

[0034] The terminals are information terminal devices such as smartphones that participants carry around the park, and they display digital game objects overlaid on the real-world scenery via their cameras. Based on game data received from the server, the terminals can detect the user's location and movements, and dynamically change the game content accordingly.

[0035] Users participate in the game using their own devices, progressing through missions while cooperating or competing with other participants. Through augmented reality (AR) game elements, users walk around the park to complete missions. Upon completion of a mission, the device sends the information to a server, and points are added to the user's account. These points can be converted into real-world assets through an electronic payment system once certain conditions are met.

[0036] As a concrete example, on a holiday afternoon, the server generates a "nature exploration mission" based on sunny weather information. Users explore the park using their devices, finding virtual plants and animals displayed in augmented reality and collecting points. The collected points are later provided to the user as a reward through an electronic payment system. In this way, active interpersonal interaction in the park and the revitalization of the local community are promoted.

[0037] The following describes the processing flow.

[0038] Step 1:

[0039] The server collects park map data, current weather, time of day, and participant profile information. Based on this, it uses an AI algorithm to generate game scenarios and missions suitable for multiple participants.

[0040] Step 2:

[0041] The server distributes the generated game data to the participants' devices. This data includes information on the placement of AR objects, mission details, and methods for interaction between participants.

[0042] Step 3:

[0043] The device uses its camera to overlay AR objects onto the surrounding scenery based on game data received from the server. A screen is then prepared that allows the user to start the game.

[0044] Step 4:

[0045] Users walk around the park with their devices, finding and interacting with mission objects displayed in augmented reality. User actions (such as taps and swipes) are detected by the device in real time.

[0046] Step 5:

[0047] The device acquires the user's current location information via GPS and dynamically adjusts the game content according to the user's movements. This provides the user with an appropriate gaming experience.

[0048] Step 6:

[0049] When a user completes a mission, the device sends the completion status to the server. Based on this information, the server adds points to the user's account.

[0050] Step 7:

[0051] The server periodically converts the accumulated points into actual reward points via an electronic payment system and reflects them in the user's account.

[0052] Step 8:

[0053] Users can check their point status through the app and redeem their earned points for use in other services. This concludes the game experience in the park.

[0054] (Example 1)

[0055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0056] Conventional augmented reality virtual experience systems have struggled to generate game content that responds to environmental changes and participants' dynamic behavior. Furthermore, they lacked sufficient interaction to encourage cooperation and competition among users, and the process of providing game results as real-world rewards was complex. This invention aims to solve these problems and provide a more engaging and interactive virtual experience.

[0057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0058] In this invention, the server includes means for generating game content using a generation AI model based on environmental information, means for distributing the generated game content to multiple electronic devices in real time, and means for displaying virtual objects superimposed on a real-world landscape based on user movement information acquired via the electronic devices. This makes it possible to provide game content that flexibly responds to environmental changes and the dynamic behavior of users, promote cooperation and competition among interactive participants, and immediately provide game results as real-world rewards.

[0059] A "park" is a public facility equipped with a natural environment, playground equipment, and walking paths, where people gather for recreation and relaxation.

[0060] An "electronic device" is a device that has the ability to process and display information, and specifically refers to smartphones, tablets, and the like.

[0061] Augmented reality technology is a technology that overlays computer-generated visual information onto the real environment, integrating reality and digital information.

[0062] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[0063] A "virtual experience" is an experience obtained by immersing participants in a digital environment separate from the real world.

[0064] "Environmental information" refers to data about the physical location and its surrounding conditions, including information such as weather, topography, and time of day.

[0065] "Game content" refers to the plan and instructions for an activity that consists of the objectives, goals, and procedures that participants will perform.

[0066] "Real-time" refers to the instantaneous processing or response that occurs the moment an operation or event takes place.

[0067] A "virtual object" is a non-physical object created by a computer and displayed through digital space or augmented reality technology.

[0068] "Movement information" refers to a collection of data such as the position, direction, and speed of a specific individual or object as it moves.

[0069] A "reward" is a monetary, physical, or digital benefit given for a particular action or achievement.

[0070] An "electronic payment system" is an infrastructure for exchanging money in a digital format via the internet.

[0071] Three main elements are involved in implementing this invention: the server, the terminal, and the user.

[0072] The server collects information from external databases and sensors to handle the park's topography and environmental information. Specifically, it uses map APIs and weather information APIs, and generates game content using a generative AI model based on this information. The generative AI model is implemented using machine learning frameworks such as TENSORFLOW® and automatically generates new missions. Specific examples of generated content include tasks such as "On a sunny day, find and record five types of virtual birds in the park." This generated content is sent to the terminal in real time using protocols such as WebSocket.

[0073] A device is an electronic device owned by the user, typically a smartphone or tablet. This device has an application using augmented reality technology installed, developed using tools such as Unity. Based on game data received from a server, this application overlays virtual objects onto the real-world scenery via the device's camera. In addition, it uses the device's built-in sensors (GPS and gyroscope) to detect the user's location and movements, analyzes the data obtained, and updates the game content as needed.

[0074] Users move around the park with their devices and participate in the game through virtual objects and missions displayed in augmented reality. For example, they might receive a prompt such as "Find five different virtual birds in the park and earn a total of 100 points or more," and aim to achieve this goal. Upon completing a mission, the device reports the progress to the server, and points are added to the user's score based on their game performance. These points are converted and provided to the user as valuable rewards through an electronic payment system.

[0075] As a result, this system provides a flexible gaming experience that adapts to the environment, encourages users to utilize the park, and enhances interaction among participants.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The server collects topographic and environmental data for the park. This input is obtained by utilizing map APIs and weather APIs to acquire the latest data in real time. Data processing includes analyzing information obtained from various sensors and converting it into the required format. As output, this information is stored in an internal database for use in the next step.

[0079] Step 2:

[0080] The server generates game content using a generative AI model based on collected terrain data and environmental information. The AI ​​model receives information about the weather, time of day, and location, and processes this data to generate specific game missions. For example, a mission such as "Find a specific virtual object at a specific location within the park" might be created and output.

[0081] Step 3:

[0082] The server delivers the generated game content to each user's device in real time. Based on the output of the generation AI model, the game data is rapidly delivered using the WebSocket protocol. As output, the game's instructions and objectives are displayed on the user's device.

[0083] Step 4:

[0084] The device overlays virtual objects onto the real-world scenery based on game data received from the server. It activates the camera and uses the obtained visual data to render 3D objects. The input consists of camera footage and data from the server, and the output is the AR view on the user's display.

[0085] Step 5:

[0086] The device uses cameras and sensors to detect the user's location and movements in real time. Data from GPS, accelerometer, and gyroscope sensors is input and analyzed to track the user's movements. Based on the detected information, the device dynamically updates the game progress and generates output that provides feedback to the user.

[0087] Step 6:

[0088] Users explore the park according to the game's instructions using the device's AR view and progress through missions. User actions are captured as input from the device, and mission completion and progress are evaluated. As output, the mission completion status is displayed on the screen, and guidance is provided to lead to the next action.

[0089] Step 7:

[0090] The device sends information to the server when the user completes a mission. The collected data (completion information) is treated as input, and points are calculated based on this. As output, points are added to the user's account, and this information is stored on the server for subsequent reward processes.

[0091] Step 8:

[0092] The server verifies that the accumulated points meet certain conditions and provides the reward via the electronic payment system. The calculated points are entered and converted into actual rewards for the user through the electronic payment system. The output is the transmission of the reward to the user's electronic payment account.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] There is a need for new methods to improve the consumer experience at facilities that provide consumer goods and to increase purchasing intent within those facilities. Furthermore, there is a significant lack of effective means to attract participants' interest and encourage sustainable use of these facilities. Therefore, the challenge lies in providing activities that consumers can enjoy participating in within the facilities, along with corresponding reward systems.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for collecting participation information from users in the facility and generating game content based on that information, means for distributing the generated game content to multiple communication devices, and means for dynamically generating game scenes based on the location information of users acquired via the communication devices. As a result, consumers can deepen their interest in products and services while having fun through games utilizing augmented reality technology within the facility, and effectively use points as rewards for commercial services.

[0098] A "facilities that provide consumer goods" refers to a physical or virtual location that directly provides goods or services to consumers.

[0099] "Communication devices" refer to electronic devices capable of sending and receiving information, including smartphones and tablets.

[0100] Augmented reality technology is a technique that overlays computer-generated information onto the real world, enabling users to interact with the real world and virtual objects.

[0101] "Participant information" refers to information about participants in a game or event, including their name, location, and behavioral patterns.

[0102] "Generating game content" means constructing playable game scenarios and tasks based on participant information and the environment at the time.

[0103] "Distributing to a communication device" means transmitting generated digital content to a designated device via the internet or other communication network.

[0104] "Location information" refers to data on physical or virtual locations identified using GPS or other location tracking technologies.

[0105] "Dynamically generating game scenes" means changing and adapting game settings and scenarios in real time according to the player's position and actions.

[0106] "Commercial service perks" refer to rewards or special treatment that customers receive when purchasing goods or services, such as discounts or bonus items.

[0107] The system for realizing this invention is designed to provide interactive games using augmented reality technology within facilities that sell consumer goods. At the core of this system is a server, which is responsible for various data processing tasks.

[0108] The server collects participation information from consumers playing games within the facility using communication devices (e.g., smartphones). This includes user location information, behavioral patterns, and areas of interest. Based on this information, the server uses a generative AI model to generate appropriate game content in real time. The generated game content is then delivered to the participants' communication devices via AR development frameworks such as ARCore and ARKit.

[0109] The device uses cameras and sensors to acquire the user's location information and accurately display AR content within the facility. Users can search for challenges generated via communication devices and compete with other participants. Points earned through the game can be converted into rewards for commercial services, encouraging users to make purchases.

[0110] As a concrete example, scanning a specific product within a facility triggers an AR game based on that product. This game progresses as the user solves problems related to the product. By accumulating and exchanging points, users can obtain specific discounts and benefits.

[0111] Examples of prompts for a generative AI model include:

[0112] "Create an AR mini-game based on a specific product scanned by a smartphone. Include interesting information and challenges to capture the user's interest."

[0113] The requirements are communicated to the server in this manner.

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The server receives location and behavioral data transmitted from participants within the facility via communication devices as input. It processes this data to understand the current context of the participating users. Through this process, the server obtains foundational data for designing game scenarios tailored to each user.

[0117] Step 2:

[0118] The server sends a prompt to the generating AI model based on the collected data, and generates specific game content. This prompt is "Generate an AR mini-game based on a specific product." The generating AI model then operates and outputs data including the game's challenges and storyline.

[0119] Step 3:

[0120] The server uses the game content obtained from the generated AI model to construct game data using an AR development framework (ARCore or ARKit). This game data is processed to include the positions of virtual objects and details of user interactions. The constructed game data is then distributed to each user's communication device.

[0121] Step 4:

[0122] The device uses game data received from the server to overlay AR content onto the camera feed. The device tracks the user's movements and location in real time and appropriately updates the AR content as the user moves within the facility. In this process, the device performs data calculations based on the user's actions and adjusts the screen display accordingly.

[0123] Step 5:

[0124] Users participate in an AR game displayed on their device and work to complete designated tasks. Information about when a user completes a mission is recorded on the device and sent to the server. As a result, users earn points, which they can later use as rewards for commercial services.

[0125] Step 6:

[0126] The server calculates points based on the received mission completion information and adds them to the user's account. If the points meet certain conditions, they are converted into a form that can be used as a reward for commercial services. The results of this process are reflected in the user's electronic trading account according to their usage.

[0127] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0128] This invention provides a system for multiple users to enjoy games using augmented reality technology with information terminal devices in a park. By further incorporating an emotion engine, this system can recognize the emotional state of each user and individually optimize the game experience.

[0129] The server collects a map of the park, participant profiles, and location information, and uses AI to generate a game scenario. This game data is then distributed to the participants' devices. The game data includes the placement of AR objects, missions, and interaction methods.

[0130] The terminals are devices such as smartphones carried by participants, and they use cameras to collect user facial expression data. This facial expression data is analyzed by an emotion engine within the terminal to identify the user's emotional state. The analysis results are sent to a server, which uses this information to adjust the game content and difficulty. For example, if the server determines that the user is bored, it will either increase the pace of the game or add a new challenge.

[0131] Meanwhile, users use their devices to explore the park, finding objects displayed in augmented reality to complete missions. Once a mission is completed, the device sends information, along with emotional data, to the server, and points are added to the user's account. These points can then be converted into something of value through an electronic payment platform.

[0132] As a concrete example, the server generates a game called "Teamwork Challenge" in a sunny park. Users move around the park with their devices and cooperate with other participants to complete missions. The emotion engine detects whether the user is feeling stressed and, if so, displays helpful hints on the device to facilitate communication with other team members. This makes the game experience more immersive and naturally promotes interaction between users.

[0133] The following describes the processing flow.

[0134] Step 1:

[0135] The server references a database containing participant registration information and uses an AI algorithm to generate a game scenario suitable for each participant based on the current conditions of the park (e.g., weather, time of day, number of participants).

[0136] Step 2:

[0137] The server sends the generated game data to each participant's device. This data includes the location of AR objects, mission details, and expected interaction details.

[0138] Step 3:

[0139] The device analyzes game data received from the server, activates the camera, and overlays AR objects onto the actual park scenery. This allows the user to start the game.

[0140] Step 4:

[0141] The device uses its camera to capture the user's facial expressions and analyzes that data in real time using its built-in emotion engine. The emotion engine identifies the user's emotional state.

[0142] Step 5:

[0143] Users move around the park with their devices, finding objects displayed in augmented reality through interaction and completing game missions.

[0144] Step 6:

[0145] The device transmits the user's location and emotional state information to the server. Based on this information, the server dynamically adjusts the game content, adaptively changing the game's difficulty and scenario progression.

[0146] Step 7:

[0147] When a user completes a mission, the device reports the result to the server. The server adds points to the user's account and converts the reward into an electronic payment system according to the user's achievement level.

[0148] Step 8:

[0149] The server uses the collected sentiment data to send information to the user's device that enhances in-game cooperation and competition to make the game more enjoyable. The device then displays this information as an interface for the user.

[0150] (Example 2)

[0151] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0152] The present invention aims to provide a system that, when offering interactive games using augmented reality technology in a park, can reflect the emotional state of individual users in real time, thereby making the game experience more personalized and optimized. It also aims to promote natural cooperation and competition among users and enhance motivation for playing the game.

[0153] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0154] In this invention, the server includes means for collecting geographic and profile information of participants and generating game scenarios using a generative AI model based on that information; means for analyzing user facial expression data acquired by the cameras of each information terminal device and identifying the user's emotional state using an emotion engine; and means for adjusting the game content and difficulty in real time based on the identified emotional state. This makes it possible to provide a game experience optimized for each user and to effectively promote cooperation and competition among users.

[0155] An "information terminal device" is a portable device that a user can carry, has functions to utilize augmented reality technology, and can collect the user's location information and emotional state.

[0156] Augmented reality technology is a technique that overlays digital data and images onto real-world images and information, allowing users to experience virtual information realistically through their vision.

[0157] A "generative AI model" is a form of artificial intelligence that can generate new game scenarios and content based on collected data.

[0158] An "emotion engine" is software or an algorithm that analyzes facial expression data acquired from a device to identify the user's emotional state.

[0159] An "electronic payment platform" is a system that converts points and other digital currencies into real-world value, enabling transactions and exchanges.

[0160] A "prompt" is a sentence containing a question or instruction that is input into a generative AI model, and is used to elicit specific answers or scenarios.

[0161] "Geographic information" refers to data that indicates the physical location of a user or device, and is obtained through GPS or other location detection technologies.

[0162] This invention is a system for multiple users in a park to enjoy games utilizing augmented reality technology using information terminal devices. The server collects users' geographical and profile information and generates new game scenarios using a generative AI model based on this information. In this process, the server gives instructions to the generative AI model using prompt statements. An example of a prompt statement is, "Design a scenario that is active and enhances teamwork."

[0163] The terminals are smartphones, tablets, and other devices owned by participating users, and they use their cameras to capture facial expression data. The captured data is analyzed by an emotion engine within the terminal to determine the user's emotional state. This analysis result is sent to a server and used to adjust the game content and difficulty in real time.

[0164] Users use their devices to freely move around the park, discovering digital objects through augmented reality (AR) technology and completing various missions. This allows users to enjoy a new experience where real space and digital information merge. Furthermore, each time a mission is completed, the device sends the information to a server, and points are added to the user's account. These points can be converted into tangible value through an electronic payment platform and exchanged for digital goods and services.

[0165] In this way, the invention combines augmented reality technology and generative AI models to provide personalized gaming experiences and enable dynamic game progression that responds to the user's emotional state.

[0166] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0167] Step 1:

[0168] The server collects geographical information within the park and user profile data. This includes GPS data and profile information previously entered by the user. Based on the entered geographical and profile information, the server determines the user's current location and prepares the basic data for formulating appropriate scenarios within the game context. The output is a dataset based on location and user attribute information.

[0169] Step 2:

[0170] The server inputs the dataset obtained in Step 1 into the AI ​​model and generates a new game scenario using prompt statements. These prompt statements instruct the AI ​​in the form of "Create AR missions suitable for each area of ​​the park." The AI ​​model processes the data and outputs a game scenario that includes the game's storyline, tasks, and AR object placement information.

[0171] Step 3:

[0172] The server delivers the generated game scenario to each user's device. The device receives the scenario information, builds a visual interface for the user, and prompts them to execute AR objects and tasks. Based on the input game scenario, it dynamically generates interactive elements to be displayed on the device. As output, it provides the user with a playable game environment in their field of view.

[0173] Step 4:

[0174] The device uses the user's camera to collect facial expression data. This data is analyzed by an emotion engine to identify the user's emotional state (e.g., happy, bored, excited). The output derived from the analyzed data is real-time feedback on the user's emotions.

[0175] Step 5:

[0176] Based on the sentiment analysis results from step 4, the server adjusts the specific game content and difficulty in real time. If it determines that the user is bored, it increases the pace of the game or offers additional challenges. This process takes sentiment data as input and generates output to optimize the existing game settings.

[0177] Step 6:

[0178] When a user completes an AR mission, the device sends its completion status to the server and simultaneously updates the user's emotional state. Based on the completion data as input, the server adds points to the user's account. The output is an increase in points, and this information is immediately fed back to the user.

[0179] Step 7:

[0180] The server converts points added to a user's account into tangible value through an electronic payment platform. Based on the input data (point information), it determines the corresponding reward or service and transfers the actual value to the user's electronic wallet or payment account as output.

[0181] (Application Example 2)

[0182] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0183] There is a growing need to provide individually optimized user experiences using advanced technology in real-world environments such as moving objects and parks. However, existing systems lack sufficient dynamic adjustments that utilize user emotions and location information, resulting in a failure to enhance user satisfaction. Furthermore, there are limited ways to effectively convert achieved scores into value.

[0184] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0185] In this invention, the server includes means for collecting participation information from people in parks or on moving objects and generating scene content based on that information; means for distributing the generated scene content to multiple information processing devices; and means for analyzing emotional states based on facial expression information of people acquired through the information processing devices and dynamically adjusting the content. This makes it possible to provide an experience optimized according to the user's emotions and to convert achievement scores into effective value in real environments such as moving objects and parks.

[0186] "Parks and mobile spaces" refer to spaces where people can gather and engage in activities, or spaces where users can gain benefits through physical experiences, such as self-driving vehicles.

[0187] An "information processing device" is an electronic device that has the function of manipulating and processing digital data, and includes smartphones, tablets, and in-car computers.

[0188] Augmented reality technology refers to the technology that enhances and modifies the real world environment using digital technology, and is also known as AR.

[0189] "Means for generating scene content" refer to technologies and algorithms used to create appropriate scenarios and experiences based on user information.

[0190] "Means of distribution" refers to the process of transferring generated digital content to various information processing devices using the internet or other means.

[0191] "Methods for analyzing emotional states based on facial expression information" refers to technologies that use facial feature data acquired by cameras and sensors to analyze a user's emotions.

[0192] "Means of dynamically adjusting content" refers to technologies that change the information and experience provided in real time according to the user's state and environment.

[0193] "Methods for converting scores into points for an electronic payment system" refers to methods of converting points earned in games or other activities into digital currency or point systems, making them available to users as economic value.

[0194] In one embodiment of this invention, the system provides an individually optimized user experience by utilizing information processing devices in parks and mobile vehicles. In particular, the system analyzes the user's emotions using augmented reality technology and an emotion engine, and dynamically adjusts the content based on that information.

[0195] The program is developed using programming languages ​​such as Python and JavaScript (registered trademark). High-performance computers or cloud services can be used for the server. The server collects participant location information and facial expression data, and generates and adjusts games and scenarios using a generative AI model. TensorFlow and PyTorch may be used for the AI ​​model.

[0196] The terminal is a smartphone or a device equipped with hardware sensors, and it uses OpenCV to detect the user's face from images acquired by the camera. The face data is sent to the server in real time, and the emotional state is analyzed. Based on the analysis results, the server adjusts and delivers the game scenario and experience content.

[0197] For example, if a user is relaxing in an autonomous vehicle, the server will select calming music in real time and provide relaxing AR content for the passenger. Furthermore, the suggested music and content can change according to the user's emotional state.

[0198] As an example of a prompt, instructions such as, "The passengers appear relaxed based on their facial expressions. Please suggest music and AR content that suits them," are input to the generating AI model, and the system then recommends and provides the most suitable content.

[0199] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0200] Step 1:

[0201] The server collects participation information from people in parks and on moving objects and stores it in a database. It takes location information and profile data of individuals as input and stores it in the database. This prepares the basic information necessary for subsequent processing steps.

[0202] Step 2:

[0203] The device activates its camera and captures the user's face to acquire facial expression data. Using the real-time captured image data as input, OpenCV is used to detect and analyze the face. This identifies the user's emotional state and generates analysis results.

[0204] Step 3:

[0205] The terminal sends the analysis results to the server. The analyzed emotion data is used as input and sent to the server. Based on the received emotion data, the server utilizes an AI model to generate appropriate game scenarios and experience content. The adjusted game data is then generated as output.

[0206] Step 4:

[0207] The server distributes the generated game data to the terminal. It uses the generated game scenario and experience content as input and delivers it to the terminal. This allows the user to experience content that is adjusted to their emotional state.

[0208] Step 5:

[0209] Users participate in the experience using their devices, and the devices collect results and new score data. The user's actions and achievements are captured as input and fed back to the server. Based on this information, the server generates scores and converts them into electronic payment points.

[0210] Step 6:

[0211] The server transfers the converted points to the user's electronic payment account. Using points that meet certain conditions as input, the system sends the points to the account via the electronic payment system. This allows the user to use the earned points as actual value.

[0212] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0213] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0214] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0215] [Second Embodiment]

[0216] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0217] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0218] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0219] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0220] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0222] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0223] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0224] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0225] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0226] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0227] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0228] This invention relates to a specific embodiment of a game system using augmented reality technology in which users participate using information terminal devices in a park.

[0229] The server uses an AI algorithm to generate game content tailored to participants, based on park map data and current environmental information. The generated game data is delivered to participants' information terminals in real time. This game data includes the location of game objects, mission details, and methods for interaction between participants.

[0230] The terminals are information terminal devices such as smartphones that participants carry around the park, and they display digital game objects overlaid on the real-world scenery via their cameras. Based on game data received from the server, the terminals can detect the user's location and movements, and dynamically change the game content accordingly.

[0231] Users participate in the game using their own devices, progressing through missions while cooperating or competing with other participants. Through augmented reality (AR) game elements, users walk around the park to complete missions. Upon completion of a mission, the device sends the information to a server, and points are added to the user's account. These points can be converted into real-world assets through an electronic payment system once certain conditions are met.

[0232] As a concrete example, on a holiday afternoon, the server generates a "nature exploration mission" based on sunny weather information. Users explore the park using their devices, finding virtual plants and animals displayed in augmented reality and collecting points. The collected points are later provided to the user as a reward through an electronic payment system. In this way, active interpersonal interaction in the park and the revitalization of the local community are promoted.

[0233] The following describes the processing flow.

[0234] Step 1:

[0235] The server collects park map data, current weather, time of day, and participant profile information. Based on this, it uses an AI algorithm to generate game scenarios and missions suitable for multiple participants.

[0236] Step 2:

[0237] The server distributes the generated game data to the participants' devices. This data includes information on the placement of AR objects, mission details, and methods for interaction between participants.

[0238] Step 3:

[0239] The device uses its camera to overlay AR objects onto the surrounding scenery based on game data received from the server. A screen is then prepared that allows the user to start the game.

[0240] Step 4:

[0241] Users walk around the park with their devices, finding and interacting with mission objects displayed in augmented reality. User actions (such as taps and swipes) are detected by the device in real time.

[0242] Step 5:

[0243] The device acquires the user's current location information via GPS and dynamically adjusts the game content according to the user's movements. This provides the user with an appropriate gaming experience.

[0244] Step 6:

[0245] When a user completes a mission, the device sends the completion status to the server. Based on this information, the server adds points to the user's account.

[0246] Step 7:

[0247] The server periodically converts the accumulated points into actual reward points via an electronic payment system and reflects them in the user's account.

[0248] Step 8:

[0249] Users can check their point status through the app and redeem their earned points for use in other services. This concludes the game experience in the park.

[0250] (Example 1)

[0251] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0252] Conventional augmented reality virtual experience systems have struggled to generate game content that responds to environmental changes and participants' dynamic behavior. Furthermore, they lacked sufficient interaction to encourage cooperation and competition among users, and the process of providing game results as real-world rewards was complex. This invention aims to solve these problems and provide a more engaging and interactive virtual experience.

[0253] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0254] In this invention, the server includes means for generating game content using a generation AI model based on environmental information, means for distributing the generated game content to multiple electronic devices in real time, and means for displaying virtual objects superimposed on a real-world landscape based on user movement information acquired via the electronic devices. This makes it possible to provide game content that flexibly responds to environmental changes and the dynamic behavior of users, promote cooperation and competition among interactive participants, and immediately provide game results as real-world rewards.

[0255] A "park" is a public facility equipped with a natural environment, playground equipment, and walking paths, where people gather for recreation and relaxation.

[0256] An "electronic device" is a device that has the ability to process and display information, and specifically refers to smartphones, tablets, and the like.

[0257] Augmented reality technology is a technology that overlays computer-generated visual information onto the real environment, integrating reality and digital information.

[0258] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[0259] A "virtual experience" is an experience obtained by immersing participants in a digital environment separate from the real world.

[0260] "Environmental information" refers to data about the physical location and its surrounding conditions, including information such as weather, topography, and time of day.

[0261] "Game content" refers to the plan and instructions for an activity that consists of the objectives, goals, and procedures that participants will perform.

[0262] "Real-time" refers to the instantaneous processing or response that occurs the moment an operation or event takes place.

[0263] A "virtual object" is a non-physical object created by a computer and displayed through digital space or augmented reality technology.

[0264] "Movement information" refers to a collection of data such as the position, direction, and speed of a specific individual or object as it moves.

[0265] A "reward" is a monetary, physical, or digital benefit given for a particular action or achievement.

[0266] An "electronic payment system" is an infrastructure for exchanging money in a digital format via the internet.

[0267] Three main elements are involved in implementing this invention: the server, the terminal, and the user.

[0268] The server collects information from external databases and sensors to handle the park's topography and environmental information. Specifically, it uses map APIs and weather information APIs, and generates game content using a generative AI model based on this information. The generative AI model is implemented using machine learning frameworks such as TensorFlow and automatically generates new missions. Specific examples of generated content include tasks such as "On a sunny day, find and record five types of virtual birds in the park." This generated content is sent to the terminal in real time using protocols such as WebSocket.

[0269] A device is an electronic device owned by the user, typically a smartphone or tablet. This device has an application using augmented reality technology installed, developed using tools such as Unity. Based on game data received from a server, this application overlays virtual objects onto the real-world scenery via the device's camera. In addition, it uses the device's built-in sensors (GPS and gyroscope) to detect the user's location and movements, analyzes the data obtained, and updates the game content as needed.

[0270] Users move around the park with their devices and participate in the game through virtual objects and missions displayed in augmented reality. For example, they might receive a prompt such as "Find five different virtual birds in the park and earn a total of 100 points or more," and aim to achieve this goal. Upon completing a mission, the device reports the progress to the server, and points are added to the user's score based on their game performance. These points are converted and provided to the user as valuable rewards through an electronic payment system.

[0271] As a result, this system provides a flexible gaming experience that adapts to the environment, encourages users to utilize the park, and enhances interaction among participants.

[0272] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0273] Step 1:

[0274] The server collects topographic and environmental data for the park. This input is obtained by utilizing map APIs and weather APIs to acquire the latest data in real time. Data processing includes analyzing information obtained from various sensors and converting it into the required format. As output, this information is stored in an internal database for use in the next step.

[0275] Step 2:

[0276] The server generates game content using a generative AI model based on collected terrain data and environmental information. The AI ​​model receives information about the weather, time of day, and location, and processes this data to generate specific game missions. For example, a mission such as "Find a specific virtual object at a specific location within the park" might be created and output.

[0277] Step 3:

[0278] The server delivers the generated game content to each user's device in real time. Based on the output of the generation AI model, the game data is rapidly delivered using the WebSocket protocol. As output, the game's instructions and objectives are displayed on the user's device.

[0279] Step 4:

[0280] The device overlays virtual objects onto the real-world scenery based on game data received from the server. It activates the camera and uses the obtained visual data to render 3D objects. The input consists of camera footage and data from the server, and the output is the AR view on the user's display.

[0281] Step 5:

[0282] The terminal uses cameras and sensors to detect the user's position and movements in real time. Data from GPS, accelerometers, and gyro sensors are input and analyzed to track the user's movements. Based on the detected information, the terminal dynamically updates the game progress and generates an output that provides feedback to the user.

[0283] Step 6:

[0284] The user uses the AR view of the terminal to explore the park according to the game content and advance the mission. The user's operations are obtained as inputs from the terminal, and the mission completion and progress are evaluated. As an output, the mission completion status is displayed on the screen, and guidance leading to the next action is provided.

[0285] Step 7:

[0286] When the user completes a mission, the terminal sends that information to the server. The collected data (completion information) is treated as an input, and points are calculated based on this. As an output, points are added to the user's account, and that information is stored on the server for subsequent reward processes.

[0287] Step 8:

[0288] The server confirms that the added points meet certain conditions and provides a reward through an electronic payment system. The calculated points are input, and a process of converting them into real rewards for the user is executed through the electronic payment system. The output is the transmission of the reward to the user's electronic payment account.

[0289] (Application Example 1)

[0290] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0291] There is a need for new methods to improve the consumer experience at facilities that provide consumer goods and to increase purchasing intent within those facilities. Furthermore, there is a significant lack of effective means to attract participants' interest and encourage sustainable use of these facilities. Therefore, the challenge lies in providing activities that consumers can enjoy participating in within the facilities, along with corresponding reward systems.

[0292] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0293] In this invention, the server includes means for collecting participation information from users in the facility and generating game content based on that information, means for distributing the generated game content to multiple communication devices, and means for dynamically generating game scenes based on the location information of users acquired via the communication devices. As a result, consumers can deepen their interest in products and services while having fun through games utilizing augmented reality technology within the facility, and effectively use points as rewards for commercial services.

[0294] A "facilities that provide consumer goods" refers to a physical or virtual location that directly provides goods or services to consumers.

[0295] "Communication devices" refer to electronic devices capable of sending and receiving information, including smartphones and tablets.

[0296] Augmented reality technology is a technique that overlays computer-generated information onto the real world, enabling users to interact with the real world and virtual objects.

[0297] "Participant information" refers to information about participants in a game or event, including their name, location, and behavioral patterns.

[0298] "Generating game content" means constructing playable game scenarios and tasks based on participant information and the environment at the time.

[0299] "Distributing to a communication device" means transmitting generated digital content to a designated device via the internet or other communication network.

[0300] "Location information" refers to data on physical or virtual locations identified using GPS or other location tracking technologies.

[0301] "Dynamically generating game scenes" means changing and adapting game settings and scenarios in real time according to the player's position and actions.

[0302] "Commercial service perks" refer to rewards or special treatment that customers receive when purchasing goods or services, such as discounts or bonus items.

[0303] The system for realizing this invention is designed to provide interactive games using augmented reality technology within facilities that sell consumer goods. At the core of this system is a server, which is responsible for various data processing tasks.

[0304] The server collects participation information from consumers playing games within the facility using communication devices (e.g., smartphones). This includes user location information, behavioral patterns, and areas of interest. Based on this information, the server uses a generative AI model to generate appropriate game content in real time. The generated game content is then delivered to the participants' communication devices via AR development frameworks such as ARCore and ARKit.

[0305] The device uses cameras and sensors to acquire the user's location information and accurately display AR content within the facility. Users can search for challenges generated via communication devices and compete with other participants. Points earned through the game can be converted into rewards for commercial services, encouraging users to make purchases.

[0306] As a specific example, when a specific product in a facility is scanned, an AR game based on that product is started. This game progresses by the user solving the content related to that product. By accumulating and exchanging points, the user can obtain specific discounts and benefits.

[0307] As an example of a prompt sentence for the generation AI model,

[0308] "Please generate an AR mini-game based on a specific product scanned by a smartphone. Please include interesting information and challenges that attract the user's interest."

[0309] The requirements are instructed to the server in the form of

[0310] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0311] Step 1:

[0312] The server receives, as input, the location information and action data transmitted by the communication device from the participants in the facility. By processing these data, the server grasps the current context of the participating users. Through this processing, the server obtains the base data for designing a game scenario suitable for each user.

[0313] Step 2:

[0314] Based on the collected data, the server transmits a prompt sentence to the generation AI model to generate specific game content. This prompt sentence has the content of "Please generate an AR mini-game based on a specific product." The generation AI model operates and outputs data including challenges and stories as game content.

[0315] Step 3:

[0316] The server uses the game content obtained from the generated AI model to construct game data using an AR development framework (ARCore or ARKit). This game data is processed to include the positions of virtual objects and details of user interactions. The constructed game data is then distributed to each user's communication device.

[0317] Step 4:

[0318] The device uses game data received from the server to overlay AR content onto the camera feed. The device tracks the user's movements and location in real time and appropriately updates the AR content as the user moves within the facility. In this process, the device performs data calculations based on the user's actions and adjusts the screen display accordingly.

[0319] Step 5:

[0320] Users participate in an AR game displayed on their device and work to complete designated tasks. Information about when a user completes a mission is recorded on the device and sent to the server. As a result, users earn points, which they can later use as rewards for commercial services.

[0321] Step 6:

[0322] The server calculates points based on the received mission completion information and adds them to the user's account. If the points meet certain conditions, they are converted into a form that can be used as a reward for commercial services. The results of this process are reflected in the user's electronic trading account according to their usage.

[0323] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0324] This invention provides a system for multiple users to enjoy games using augmented reality technology with information terminal devices in a park. By further incorporating an emotion engine, this system can recognize the emotional state of each user and individually optimize the game experience.

[0325] The server collects a map of the park, participant profiles, and location information, and uses AI to generate a game scenario. This game data is then distributed to the participants' devices. The game data includes the placement of AR objects, missions, and interaction methods.

[0326] The terminals are devices such as smartphones carried by participants, and they use cameras to collect user facial expression data. This facial expression data is analyzed by an emotion engine within the terminal to identify the user's emotional state. The analysis results are sent to a server, which uses this information to adjust the game content and difficulty. For example, if the server determines that the user is bored, it will either increase the pace of the game or add a new challenge.

[0327] Meanwhile, users use their devices to explore the park, finding objects displayed in augmented reality to complete missions. Once a mission is completed, the device sends information, along with emotional data, to the server, and points are added to the user's account. These points can then be converted into something of value through an electronic payment platform.

[0328] As a concrete example, the server generates a game called "Teamwork Challenge" in a sunny park. Users move around the park with their devices and cooperate with other participants to complete missions. The emotion engine detects whether the user is feeling stressed and, if so, displays helpful hints on the device to facilitate communication with other team members. This makes the game experience more immersive and naturally promotes interaction between users.

[0329] The following describes the processing flow.

[0330] Step 1:

[0331] The server references a database containing participant registration information and uses an AI algorithm to generate a game scenario suitable for each participant based on the current conditions of the park (e.g., weather, time of day, number of participants).

[0332] Step 2:

[0333] The server sends the generated game data to each participant's device. This data includes the location of AR objects, mission details, and expected interaction details.

[0334] Step 3:

[0335] The device analyzes game data received from the server, activates the camera, and overlays AR objects onto the actual park scenery. This allows the user to start the game.

[0336] Step 4:

[0337] The device uses its camera to capture the user's facial expressions and analyzes that data in real time using its built-in emotion engine. The emotion engine identifies the user's emotional state.

[0338] Step 5:

[0339] Users move around the park with their devices, finding objects displayed in augmented reality through interaction and completing game missions.

[0340] Step 6:

[0341] The device transmits the user's location and emotional state information to the server. Based on this information, the server dynamically adjusts the game content, adaptively changing the game's difficulty and scenario progression.

[0342] Step 7:

[0343] When a user completes a mission, the device reports the result to the server. The server adds points to the user's account and converts the reward into an electronic payment system according to the user's achievement level.

[0344] Step 8:

[0345] The server uses the collected sentiment data to send information to the user's device that enhances in-game cooperation and competition to make the game more enjoyable. The device then displays this information as an interface for the user.

[0346] (Example 2)

[0347] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0348] The present invention aims to provide a system that, when offering interactive games using augmented reality technology in a park, can reflect the emotional state of individual users in real time, thereby making the game experience more personalized and optimized. It also aims to promote natural cooperation and competition among users and enhance motivation for playing the game.

[0349] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0350] In this invention, the server includes means for collecting geographic and profile information of participants and generating game scenarios using a generative AI model based on that information; means for analyzing user facial expression data acquired by the cameras of each information terminal device and identifying the user's emotional state using an emotion engine; and means for adjusting the game content and difficulty in real time based on the identified emotional state. This makes it possible to provide a game experience optimized for each user and to effectively promote cooperation and competition among users.

[0351] An "information terminal device" is a portable device that a user can carry, has functions to utilize augmented reality technology, and can collect the user's location information and emotional state.

[0352] Augmented reality technology is a technique that overlays digital data and images onto real-world images and information, allowing users to experience virtual information realistically through their vision.

[0353] A "generative AI model" is a form of artificial intelligence that can generate new game scenarios and content based on collected data.

[0354] An "emotion engine" is software or an algorithm that analyzes facial expression data acquired from a device to identify the user's emotional state.

[0355] An "electronic payment platform" is a system that converts points and other digital currencies into real-world value, enabling transactions and exchanges.

[0356] A "prompt" is a sentence containing a question or instruction that is input into a generative AI model, and is used to elicit specific answers or scenarios.

[0357] "Geographic information" refers to data that indicates the physical location of a user or device, and is obtained through GPS or other location detection technologies.

[0358] This invention is a system for multiple users in a park to enjoy games utilizing augmented reality technology using information terminal devices. The server collects users' geographical and profile information and generates new game scenarios using a generative AI model based on this information. In this process, the server gives instructions to the generative AI model using prompt statements. An example of a prompt statement is, "Design a scenario that is active and enhances teamwork."

[0359] The terminals are smartphones, tablets, and other devices owned by participating users, and they use their cameras to capture facial expression data. The captured data is analyzed by an emotion engine within the terminal to determine the user's emotional state. This analysis result is sent to a server and used to adjust the game content and difficulty in real time.

[0360] Users use their devices to freely move around the park, discovering digital objects through augmented reality (AR) technology and completing various missions. This allows users to enjoy a new experience where real space and digital information merge. Furthermore, each time a mission is completed, the device sends the information to a server, and points are added to the user's account. These points can be converted into tangible value through an electronic payment platform and exchanged for digital goods and services.

[0361] In this way, the invention combines augmented reality technology and generative AI models to provide personalized gaming experiences and enable dynamic game progression that responds to the user's emotional state.

[0362] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0363] Step 1:

[0364] The server collects geographical information within the park and user profile data. This includes GPS data and profile information previously entered by the user. Based on the entered geographical and profile information, the server determines the user's current location and prepares the basic data for formulating appropriate scenarios within the game context. The output is a dataset based on location and user attribute information.

[0365] Step 2:

[0366] The server inputs the dataset obtained in Step 1 into the AI ​​model and generates a new game scenario using prompt statements. These prompt statements instruct the AI ​​in the form of "Create AR missions suitable for each area of ​​the park." The AI ​​model processes the data and outputs a game scenario that includes the game's storyline, tasks, and AR object placement information.

[0367] Step 3:

[0368] The server delivers the generated game scenario to each user's device. The device receives the scenario information, builds a visual interface for the user, and prompts them to execute AR objects and tasks. Based on the input game scenario, it dynamically generates interactive elements to be displayed on the device. As output, it provides the user with a playable game environment in their field of view.

[0369] Step 4:

[0370] The device uses the user's camera to collect facial expression data. This data is analyzed by an emotion engine to identify the user's emotional state (e.g., happy, bored, excited). The output derived from the analyzed data is real-time feedback on the user's emotions.

[0371] Step 5:

[0372] Based on the sentiment analysis results from step 4, the server adjusts the specific game content and difficulty in real time. If it determines that the user is bored, it increases the pace of the game or offers additional challenges. This process takes sentiment data as input and generates output to optimize the existing game settings.

[0373] Step 6:

[0374] When a user completes an AR mission, the device sends its completion status to the server and simultaneously updates the user's emotional state. Based on the completion data as input, the server adds points to the user's account. The output is an increase in points, and this information is immediately fed back to the user.

[0375] Step 7:

[0376] The server converts points added to a user's account into tangible value through an electronic payment platform. Based on the input data (point information), it determines the corresponding reward or service and transfers the actual value to the user's electronic wallet or payment account as output.

[0377] (Application Example 2)

[0378] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0379] There is a growing need to provide individually optimized user experiences using advanced technology in real-world environments such as moving objects and parks. However, existing systems lack sufficient dynamic adjustments that utilize user emotions and location information, resulting in a failure to enhance user satisfaction. Furthermore, there are limited ways to effectively convert achieved scores into value.

[0380] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0381] In this invention, the server includes means for collecting participation information from people in parks or on moving objects and generating scene content based on that information; means for distributing the generated scene content to multiple information processing devices; and means for analyzing emotional states based on facial expression information of people acquired through the information processing devices and dynamically adjusting the content. This makes it possible to provide an experience optimized according to the user's emotions and to convert achievement scores into effective value in real environments such as moving objects and parks.

[0382] "Parks and mobile spaces" refer to spaces where people can gather and engage in activities, or spaces where users can gain benefits through physical experiences, such as self-driving vehicles.

[0383] An "information processing device" is an electronic device that has the function of manipulating and processing digital data, and includes smartphones, tablets, and in-car computers.

[0384] Augmented reality technology refers to the technology that enhances and modifies the real world environment using digital technology, and is also known as AR.

[0385] "Means for generating scene content" refer to technologies and algorithms used to create appropriate scenarios and experiences based on user information.

[0386] "Means of distribution" refers to the process of transferring generated digital content to various information processing devices using the internet or other means.

[0387] "Methods for analyzing emotional states based on facial expression information" refers to technologies that use facial feature data acquired by cameras and sensors to analyze a user's emotions.

[0388] "Means of dynamically adjusting content" refers to technologies that change the information and experience provided in real time according to the user's state and environment.

[0389] "Methods for converting scores into points for an electronic payment system" refers to methods of converting points earned in games or other activities into digital currency or point systems, making them available to users as economic value.

[0390] In one embodiment of this invention, the system provides an individually optimized user experience by utilizing information processing devices in parks and mobile vehicles. In particular, the system analyzes the user's emotions using augmented reality technology and an emotion engine, and dynamically adjusts the content based on that information.

[0391] The program is developed using programming languages ​​such as Python and JavaScript. High-performance computers or cloud services can be used for the server. The server collects participant location information and facial expression data, and generates and adjusts games and scenarios using a generative AI model. TensorFlow and PyTorch may be used for the AI ​​model.

[0392] The terminal is a smartphone or a device equipped with hardware sensors, and it uses OpenCV to detect the user's face from images acquired by the camera. The face data is sent to the server in real time, and the emotional state is analyzed. Based on the analysis results, the server adjusts and delivers the game scenario and experience content.

[0393] For example, if a user is relaxing in an autonomous vehicle, the server will select calming music in real time and provide relaxing AR content for the passenger. Furthermore, the suggested music and content can change according to the user's emotional state.

[0394] As an example of a prompt, instructions such as, "The passengers appear relaxed based on their facial expressions. Please suggest music and AR content that suits them," are input to the generating AI model, and the system then recommends and provides the most suitable content.

[0395] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0396] Step 1:

[0397] The server collects participation information from people in parks and on moving objects and stores it in a database. It takes location information and profile data of individuals as input and stores it in the database. This prepares the basic information necessary for subsequent processing steps.

[0398] Step 2:

[0399] The device activates its camera and captures the user's face to acquire facial expression data. Using the real-time captured image data as input, OpenCV is used to detect and analyze the face. This identifies the user's emotional state and generates analysis results.

[0400] Step 3:

[0401] The terminal sends the analysis results to the server. The analyzed emotion data is used as input and sent to the server. Based on the received emotion data, the server utilizes an AI model to generate appropriate game scenarios and experience content. The adjusted game data is then generated as output.

[0402] Step 4:

[0403] The server distributes the generated game data to the terminal. It uses the generated game scenario and experience content as input and delivers it to the terminal. This allows the user to experience content that is adjusted to their emotional state.

[0404] Step 5:

[0405] Users participate in the experience using their devices, and the devices collect results and new score data. The user's actions and achievements are captured as input and fed back to the server. Based on this information, the server generates scores and converts them into electronic payment points.

[0406] Step 6:

[0407] The server transfers the converted points to the user's electronic payment account. Using points that meet certain conditions as input, the system sends the points to the account via the electronic payment system. This allows the user to use the earned points as actual value.

[0408] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0409] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0410] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0411] [Third Embodiment]

[0412] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0413] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0414] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0415] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0416] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0417] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0418] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0419] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0420] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0421] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0422] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0423] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0424] This invention relates to a specific embodiment of a game system using augmented reality technology in which users participate using information terminal devices in a park.

[0425] The server uses an AI algorithm to generate game content tailored to participants, based on park map data and current environmental information. The generated game data is delivered to participants' information terminals in real time. This game data includes the location of game objects, mission details, and methods for interaction between participants.

[0426] The terminals are information terminal devices such as smartphones that participants carry around the park, and they display digital game objects overlaid on the real-world scenery via their cameras. Based on game data received from the server, the terminals can detect the user's location and movements, and dynamically change the game content accordingly.

[0427] Users participate in the game using their own devices, progressing through missions while cooperating or competing with other participants. Through augmented reality (AR) game elements, users walk around the park to complete missions. Upon completion of a mission, the device sends the information to a server, and points are added to the user's account. These points can be converted into real-world assets through an electronic payment system once certain conditions are met.

[0428] As a concrete example, on a holiday afternoon, the server generates a "nature exploration mission" based on sunny weather information. Users explore the park using their devices, finding virtual plants and animals displayed in augmented reality and collecting points. The collected points are later provided to the user as a reward through an electronic payment system. In this way, active interpersonal interaction in the park and the revitalization of the local community are promoted.

[0429] The following describes the processing flow.

[0430] Step 1:

[0431] The server collects park map data, current weather, time of day, and participant profile information. Based on this, it uses an AI algorithm to generate game scenarios and missions suitable for multiple participants.

[0432] Step 2:

[0433] The server distributes the generated game data to the participants' devices. This data includes information on the placement of AR objects, mission details, and methods for interaction between participants.

[0434] Step 3:

[0435] The device uses its camera to overlay AR objects onto the surrounding scenery based on game data received from the server. A screen is then prepared that allows the user to start the game.

[0436] Step 4:

[0437] Users walk around the park with their devices, finding and interacting with mission objects displayed in augmented reality. User actions (such as taps and swipes) are detected by the device in real time.

[0438] Step 5:

[0439] The device acquires the user's current location information via GPS and dynamically adjusts the game content according to the user's movements. This provides the user with an appropriate gaming experience.

[0440] Step 6:

[0441] When a user completes a mission, the device sends the completion status to the server. Based on this information, the server adds points to the user's account.

[0442] Step 7:

[0443] The server periodically converts the accumulated points into actual reward points via an electronic payment system and reflects them in the user's account.

[0444] Step 8:

[0445] Users can check their point status through the app and redeem their earned points for use in other services. This concludes the game experience in the park.

[0446] (Example 1)

[0447] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0448] Conventional augmented reality virtual experience systems have struggled to generate game content that responds to environmental changes and participants' dynamic behavior. Furthermore, they lacked sufficient interaction to encourage cooperation and competition among users, and the process of providing game results as real-world rewards was complex. This invention aims to solve these problems and provide a more engaging and interactive virtual experience.

[0449] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0450] In this invention, the server includes means for generating game content using a generation AI model based on environmental information, means for distributing the generated game content to multiple electronic devices in real time, and means for displaying virtual objects superimposed on a real-world landscape based on user movement information acquired via the electronic devices. This makes it possible to provide game content that flexibly responds to environmental changes and the dynamic behavior of users, promote cooperation and competition among interactive participants, and immediately provide game results as real-world rewards.

[0451] A "park" is a public facility equipped with a natural environment, playground equipment, and walking paths, where people gather for recreation and relaxation.

[0452] An "electronic device" is a device that has the ability to process and display information, and specifically refers to smartphones, tablets, and the like.

[0453] Augmented reality technology is a technology that overlays computer-generated visual information onto the real environment, integrating reality and digital information.

[0454] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[0455] A "virtual experience" is an experience obtained by immersing participants in a digital environment separate from the real world.

[0456] "Environmental information" refers to data about the physical location and its surrounding conditions, including information such as weather, topography, and time of day.

[0457] "Game content" refers to the plan and instructions for an activity that consists of the objectives, goals, and procedures that participants will perform.

[0458] "Real-time" refers to the instantaneous processing or response that occurs the moment an operation or event takes place.

[0459] A "virtual object" is a non-physical object created by a computer and displayed through digital space or augmented reality technology.

[0460] "Movement information" refers to a collection of data such as the position, direction, and speed of a specific individual or object as it moves.

[0461] A "reward" is a monetary, physical, or digital benefit given for a particular action or achievement.

[0462] An "electronic payment system" is an infrastructure for exchanging money in a digital format via the internet.

[0463] Three main elements are involved in implementing this invention: the server, the terminal, and the user.

[0464] The server collects information from external databases and sensors to handle the park's topography and environmental information. Specifically, it uses map APIs and weather information APIs, and generates game content using a generative AI model based on this information. The generative AI model is implemented using machine learning frameworks such as TensorFlow and automatically generates new missions. Specific examples of generated content include tasks such as "On a sunny day, find and record five types of virtual birds in the park." This generated content is sent to the terminal in real time using protocols such as WebSocket.

[0465] A device is an electronic device owned by the user, typically a smartphone or tablet. This device has an application using augmented reality technology installed, developed using tools such as Unity. Based on game data received from a server, this application overlays virtual objects onto the real-world scenery via the device's camera. In addition, it uses the device's built-in sensors (GPS and gyroscope) to detect the user's location and movements, analyzes the data obtained, and updates the game content as needed.

[0466] Users move around the park with their devices and participate in the game through virtual objects and missions displayed in augmented reality. For example, they might receive a prompt such as "Find five different virtual birds in the park and earn a total of 100 points or more," and aim to achieve this goal. Upon completing a mission, the device reports the progress to the server, and points are added to the user's score based on their game performance. These points are converted and provided to the user as valuable rewards through an electronic payment system.

[0467] As a result, this system provides a flexible gaming experience that adapts to the environment, encourages users to utilize the park, and enhances interaction among participants.

[0468] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0469] Step 1:

[0470] The server collects topographic and environmental data for the park. This input is obtained by utilizing map APIs and weather APIs to acquire the latest data in real time. Data processing includes analyzing information obtained from various sensors and converting it into the required format. As output, this information is stored in an internal database for use in the next step.

[0471] Step 2:

[0472] The server generates game content using a generative AI model based on collected terrain data and environmental information. The AI ​​model receives information about the weather, time of day, and location, and processes this data to generate specific game missions. For example, a mission such as "Find a specific virtual object at a specific location within the park" might be created and output.

[0473] Step 3:

[0474] The server delivers the generated game content to each user's device in real time. Based on the output of the generation AI model, the game data is rapidly delivered using the WebSocket protocol. As output, the game's instructions and objectives are displayed on the user's device.

[0475] Step 4:

[0476] The device overlays virtual objects onto the real-world scenery based on game data received from the server. It activates the camera and uses the obtained visual data to render 3D objects. The input consists of camera footage and data from the server, and the output is the AR view on the user's display.

[0477] Step 5:

[0478] The device uses cameras and sensors to detect the user's location and movements in real time. Data from GPS, accelerometer, and gyroscope sensors is input and analyzed to track the user's movements. Based on the detected information, the device dynamically updates the game progress and generates output that provides feedback to the user.

[0479] Step 6:

[0480] Users explore the park according to the game's instructions using the device's AR view and progress through missions. User actions are captured as input from the device, and mission completion and progress are evaluated. As output, the mission completion status is displayed on the screen, and guidance is provided to lead to the next action.

[0481] Step 7:

[0482] The device sends information to the server when the user completes a mission. The collected data (completion information) is treated as input, and points are calculated based on this. As output, points are added to the user's account, and this information is stored on the server for subsequent reward processes.

[0483] Step 8:

[0484] The server verifies that the accumulated points meet certain conditions and provides the reward via the electronic payment system. The calculated points are entered and converted into actual rewards for the user through the electronic payment system. The output is the transmission of the reward to the user's electronic payment account.

[0485] (Application Example 1)

[0486] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0487] There is a need for new methods to improve the consumer experience at facilities that provide consumer goods and to increase purchasing intent within those facilities. Furthermore, there is a significant lack of effective means to attract participants' interest and encourage sustainable use of these facilities. Therefore, the challenge lies in providing activities that consumers can enjoy participating in within the facilities, along with corresponding reward systems.

[0488] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0489] In this invention, the server includes means for collecting participation information from users in the facility and generating game content based on that information, means for distributing the generated game content to multiple communication devices, and means for dynamically generating game scenes based on the location information of users acquired via the communication devices. As a result, consumers can deepen their interest in products and services while having fun through games utilizing augmented reality technology within the facility, and effectively use points as rewards for commercial services.

[0490] A "facilities that provide consumer goods" refers to a physical or virtual location that directly provides goods or services to consumers.

[0491] "Communication devices" refer to electronic devices capable of sending and receiving information, including smartphones and tablets.

[0492] Augmented reality technology is a technique that overlays computer-generated information onto the real world, enabling users to interact with the real world and virtual objects.

[0493] "Participant information" refers to information about participants in a game or event, including their name, location, and behavioral patterns.

[0494] "Generating game content" means constructing playable game scenarios and tasks based on participant information and the environment at the time.

[0495] "Distributing to a communication device" means transmitting generated digital content to a designated device via the internet or other communication network.

[0496] "Location information" refers to data on physical or virtual locations identified using GPS or other location tracking technologies.

[0497] "Dynamically generating game scenes" means changing and adapting game settings and scenarios in real time according to the player's position and actions.

[0498] "Commercial service perks" refer to rewards or special treatment that customers receive when purchasing goods or services, such as discounts or bonus items.

[0499] The system for realizing this invention is designed to provide interactive games using augmented reality technology within facilities that sell consumer goods. At the core of this system is a server, which is responsible for various data processing tasks.

[0500] The server collects participation information from consumers playing games within the facility using communication devices (e.g., smartphones). This includes user location information, behavioral patterns, and areas of interest. Based on this information, the server uses a generative AI model to generate appropriate game content in real time. The generated game content is then delivered to the participants' communication devices via AR development frameworks such as ARCore and ARKit.

[0501] The device uses cameras and sensors to acquire the user's location information and accurately display AR content within the facility. Users can search for challenges generated via communication devices and compete with other participants. Points earned through the game can be converted into rewards for commercial services, encouraging users to make purchases.

[0502] As a concrete example, scanning a specific product within a facility triggers an AR game based on that product. This game progresses as the user solves problems related to the product. By accumulating and exchanging points, users can obtain specific discounts and benefits.

[0503] Examples of prompts for a generative AI model include:

[0504] "Create an AR mini-game based on a specific product scanned by a smartphone. Include interesting information and challenges to capture the user's interest."

[0505] The requirements are communicated to the server in this manner.

[0506] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0507] Step 1:

[0508] The server receives location and behavioral data transmitted from participants within the facility via communication devices as input. It processes this data to understand the current context of the participating users. Through this process, the server obtains foundational data for designing game scenarios tailored to each user.

[0509] Step 2:

[0510] The server sends a prompt to the generating AI model based on the collected data, and generates specific game content. This prompt is "Generate an AR mini-game based on a specific product." The generating AI model then operates and outputs data including the game's challenges and storyline.

[0511] Step 3:

[0512] The server uses the game content obtained from the generated AI model to construct game data using an AR development framework (ARCore or ARKit). This game data is processed to include the positions of virtual objects and details of user interactions. The constructed game data is then distributed to each user's communication device.

[0513] Step 4:

[0514] The device uses game data received from the server to overlay AR content onto the camera feed. The device tracks the user's movements and location in real time and appropriately updates the AR content as the user moves within the facility. In this process, the device performs data calculations based on the user's actions and adjusts the screen display accordingly.

[0515] Step 5:

[0516] Users participate in an AR game displayed on their device and work to complete designated tasks. Information about when a user completes a mission is recorded on the device and sent to the server. As a result, users earn points, which they can later use as rewards for commercial services.

[0517] Step 6:

[0518] The server calculates points based on the received mission completion information and adds them to the user's account. If the points meet certain conditions, they are converted into a form that can be used as a reward for commercial services. The results of this process are reflected in the user's electronic trading account according to their usage.

[0519] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0520] This invention provides a system for multiple users to enjoy games using augmented reality technology with information terminal devices in a park. By further incorporating an emotion engine, this system can recognize the emotional state of each user and individually optimize the game experience.

[0521] The server collects a map of the park, participant profiles, and location information, and uses AI to generate a game scenario. This game data is then distributed to the participants' devices. The game data includes the placement of AR objects, missions, and interaction methods.

[0522] The terminals are devices such as smartphones carried by participants, and they use cameras to collect user facial expression data. This facial expression data is analyzed by an emotion engine within the terminal to identify the user's emotional state. The analysis results are sent to a server, which uses this information to adjust the game content and difficulty. For example, if the server determines that the user is bored, it will either increase the pace of the game or add a new challenge.

[0523] Meanwhile, users use their devices to explore the park, finding objects displayed in augmented reality to complete missions. Once a mission is completed, the device sends information, along with emotional data, to the server, and points are added to the user's account. These points can then be converted into something of value through an electronic payment platform.

[0524] As a concrete example, the server generates a game called "Teamwork Challenge" in a sunny park. Users move around the park with their devices and cooperate with other participants to complete missions. The emotion engine detects whether the user is feeling stressed and, if so, displays helpful hints on the device to facilitate communication with other team members. This makes the game experience more immersive and naturally promotes interaction between users.

[0525] The following describes the processing flow.

[0526] Step 1:

[0527] The server references a database containing participant registration information and uses an AI algorithm to generate a game scenario suitable for each participant based on the current conditions of the park (e.g., weather, time of day, number of participants).

[0528] Step 2:

[0529] The server sends the generated game data to each participant's device. This data includes the location of AR objects, mission details, and expected interaction details.

[0530] Step 3:

[0531] The device analyzes game data received from the server, activates the camera, and overlays AR objects onto the actual park scenery. This allows the user to start the game.

[0532] Step 4:

[0533] The device uses its camera to capture the user's facial expressions and analyzes that data in real time using its built-in emotion engine. The emotion engine identifies the user's emotional state.

[0534] Step 5:

[0535] Users move around the park with their devices, finding objects displayed in augmented reality through interaction and completing game missions.

[0536] Step 6:

[0537] The device transmits the user's location and emotional state information to the server. Based on this information, the server dynamically adjusts the game content, adaptively changing the game's difficulty and scenario progression.

[0538] Step 7:

[0539] When a user completes a mission, the device reports the result to the server. The server adds points to the user's account and converts the reward into an electronic payment system according to the user's achievement level.

[0540] Step 8:

[0541] The server uses the collected sentiment data to send information to the user's device that enhances in-game cooperation and competition to make the game more enjoyable. The device then displays this information as an interface for the user.

[0542] (Example 2)

[0543] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0544] The present invention aims to provide a system that, when offering interactive games using augmented reality technology in a park, can reflect the emotional state of individual users in real time, thereby making the game experience more personalized and optimized. It also aims to promote natural cooperation and competition among users and enhance motivation for playing the game.

[0545] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0546] In this invention, the server includes means for collecting geographic and profile information of participants and generating game scenarios using a generative AI model based on that information; means for analyzing user facial expression data acquired by the cameras of each information terminal device and identifying the user's emotional state using an emotion engine; and means for adjusting the game content and difficulty in real time based on the identified emotional state. This makes it possible to provide a game experience optimized for each user and to effectively promote cooperation and competition among users.

[0547] An "information terminal device" is a portable device that a user can carry, has functions to utilize augmented reality technology, and can collect the user's location information and emotional state.

[0548] Augmented reality technology is a technique that overlays digital data and images onto real-world images and information, allowing users to experience virtual information realistically through their vision.

[0549] A "generative AI model" is a form of artificial intelligence that can generate new game scenarios and content based on collected data.

[0550] An "emotion engine" is software or an algorithm that analyzes facial expression data acquired from a device to identify the user's emotional state.

[0551] An "electronic payment platform" is a system that converts points and other digital currencies into real-world value, enabling transactions and exchanges.

[0552] A "prompt" is a sentence containing a question or instruction that is input into a generative AI model, and is used to elicit specific answers or scenarios.

[0553] "Geographic information" refers to data that indicates the physical location of a user or device, and is obtained through GPS or other location detection technologies.

[0554] This invention is a system for multiple users in a park to enjoy games utilizing augmented reality technology using information terminal devices. The server collects users' geographical and profile information and generates new game scenarios using a generative AI model based on this information. In this process, the server gives instructions to the generative AI model using prompt statements. An example of a prompt statement is, "Design a scenario that is active and enhances teamwork."

[0555] The terminals are smartphones, tablets, and other devices owned by participating users, and they use their cameras to capture facial expression data. The captured data is analyzed by an emotion engine within the terminal to determine the user's emotional state. This analysis result is sent to a server and used to adjust the game content and difficulty in real time.

[0556] Users use their devices to freely move around the park, discovering digital objects through augmented reality (AR) technology and completing various missions. This allows users to enjoy a new experience where real space and digital information merge. Furthermore, each time a mission is completed, the device sends the information to a server, and points are added to the user's account. These points can be converted into tangible value through an electronic payment platform and exchanged for digital goods and services.

[0557] In this way, the invention combines augmented reality technology and generative AI models to provide personalized gaming experiences and enable dynamic game progression that responds to the user's emotional state.

[0558] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0559] Step 1:

[0560] The server collects geographical information within the park and user profile data. This includes GPS data and profile information previously entered by the user. Based on the entered geographical and profile information, the server determines the user's current location and prepares the basic data for formulating appropriate scenarios within the game context. The output is a dataset based on location and user attribute information.

[0561] Step 2:

[0562] The server inputs the dataset obtained in Step 1 into the AI ​​model and generates a new game scenario using prompt statements. These prompt statements instruct the AI ​​in the form of "Create AR missions suitable for each area of ​​the park." The AI ​​model processes the data and outputs a game scenario that includes the game's storyline, tasks, and AR object placement information.

[0563] Step 3:

[0564] The server delivers the generated game scenario to each user's device. The device receives the scenario information, builds a visual interface for the user, and prompts them to execute AR objects and tasks. Based on the input game scenario, it dynamically generates interactive elements to be displayed on the device. As output, it provides the user with a playable game environment in their field of view.

[0565] Step 4:

[0566] The device uses the user's camera to collect facial expression data. This data is analyzed by an emotion engine to identify the user's emotional state (e.g., happy, bored, excited). The output derived from the analyzed data is real-time feedback on the user's emotions.

[0567] Step 5:

[0568] Based on the sentiment analysis results from step 4, the server adjusts the specific game content and difficulty in real time. If it determines that the user is bored, it increases the pace of the game or offers additional challenges. This process takes sentiment data as input and generates output to optimize the existing game settings.

[0569] Step 6:

[0570] When a user completes an AR mission, the device sends its completion status to the server and simultaneously updates the user's emotional state. Based on the completion data as input, the server adds points to the user's account. The output is an increase in points, and this information is immediately fed back to the user.

[0571] Step 7:

[0572] The server converts points added to a user's account into tangible value through an electronic payment platform. Based on the input data (point information), it determines the corresponding reward or service and transfers the actual value to the user's electronic wallet or payment account as output.

[0573] (Application Example 2)

[0574] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0575] There is a growing need to provide individually optimized user experiences using advanced technology in real-world environments such as moving objects and parks. However, existing systems lack sufficient dynamic adjustments that utilize user emotions and location information, resulting in a failure to enhance user satisfaction. Furthermore, there are limited ways to effectively convert achieved scores into value.

[0576] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0577] In this invention, the server includes means for collecting participation information from people in parks or on moving objects and generating scene content based on that information; means for distributing the generated scene content to multiple information processing devices; and means for analyzing emotional states based on facial expression information of people acquired through the information processing devices and dynamically adjusting the content. This makes it possible to provide an experience optimized according to the user's emotions and to convert achievement scores into effective value in real environments such as moving objects and parks.

[0578] "Parks and mobile spaces" refer to spaces where people can gather and engage in activities, or spaces where users can gain benefits through physical experiences, such as self-driving vehicles.

[0579] An "information processing device" is an electronic device that has the function of manipulating and processing digital data, and includes smartphones, tablets, and in-car computers.

[0580] Augmented reality technology refers to the technology that enhances and modifies the real world environment using digital technology, and is also known as AR.

[0581] "Means for generating scene content" refer to technologies and algorithms used to create appropriate scenarios and experiences based on user information.

[0582] "Means of distribution" refers to the process of transferring generated digital content to various information processing devices using the internet or other means.

[0583] "Methods for analyzing emotional states based on facial expression information" refers to technologies that use facial feature data acquired by cameras and sensors to analyze a user's emotions.

[0584] "Means of dynamically adjusting content" refers to technologies that change the information and experience provided in real time according to the user's state and environment.

[0585] "Methods for converting scores into points for an electronic payment system" refers to methods of converting points earned in games or other activities into digital currency or point systems, making them available to users as economic value.

[0586] In one embodiment of this invention, the system provides an individually optimized user experience by utilizing information processing devices in parks and mobile vehicles. In particular, the system analyzes the user's emotions using augmented reality technology and an emotion engine, and dynamically adjusts the content based on that information.

[0587] The program is developed using programming languages ​​such as Python and JavaScript. High-performance computers or cloud services can be used for the server. The server collects participant location information and facial expression data, and generates and adjusts games and scenarios using a generative AI model. TensorFlow and PyTorch may be used for the AI ​​model.

[0588] The terminal is a smartphone or a device equipped with hardware sensors, and it uses OpenCV to detect the user's face from images acquired by the camera. The face data is sent to the server in real time, and the emotional state is analyzed. Based on the analysis results, the server adjusts and delivers the game scenario and experience content.

[0589] For example, if a user is relaxing in an autonomous vehicle, the server will select calming music in real time and provide relaxing AR content for the passenger. Furthermore, the suggested music and content can change according to the user's emotional state.

[0590] As an example of a prompt, instructions such as, "The passengers appear relaxed based on their facial expressions. Please suggest music and AR content that suits them," are input to the generating AI model, and the system then recommends and provides the most suitable content.

[0591] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0592] Step 1:

[0593] The server collects participation information from people in parks and on moving objects and stores it in a database. It takes location information and profile data of individuals as input and stores it in the database. This prepares the basic information necessary for subsequent processing steps.

[0594] Step 2:

[0595] The device activates its camera and captures the user's face to acquire facial expression data. Using the real-time captured image data as input, OpenCV is used to detect and analyze the face. This identifies the user's emotional state and generates analysis results.

[0596] Step 3:

[0597] The terminal sends the analysis results to the server. The analyzed emotion data is used as input and sent to the server. Based on the received emotion data, the server utilizes an AI model to generate appropriate game scenarios and experience content. The adjusted game data is then generated as output.

[0598] Step 4:

[0599] The server distributes the generated game data to the terminal. It uses the generated game scenario and experience content as input and delivers it to the terminal. This allows the user to experience content that is adjusted to their emotional state.

[0600] Step 5:

[0601] Users participate in the experience using their devices, and the devices collect results and new score data. The user's actions and achievements are captured as input and fed back to the server. Based on this information, the server generates scores and converts them into electronic payment points.

[0602] Step 6:

[0603] The server transfers the converted points to the user's electronic payment account. Using points that meet certain conditions as input, the system sends the points to the account via the electronic payment system. This allows the user to use the earned points as actual value.

[0604] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0605] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0606] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0607] [Fourth Embodiment]

[0608] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0609] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0610] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0611] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0612] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0613] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0614] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0615] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0616] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0617] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0618] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0619] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0620] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] This invention relates to a specific embodiment of a game system using augmented reality technology in which users participate using information terminal devices in a park.

[0622] The server uses an AI algorithm to generate game content tailored to participants, based on park map data and current environmental information. The generated game data is delivered to participants' information terminals in real time. This game data includes the location of game objects, mission details, and methods for interaction between participants.

[0623] The terminals are information terminal devices such as smartphones that participants carry around the park, and they display digital game objects overlaid on the real-world scenery via their cameras. Based on game data received from the server, the terminals can detect the user's location and movements, and dynamically change the game content accordingly.

[0624] Users participate in the game using their own devices, progressing through missions while cooperating or competing with other participants. Through augmented reality (AR) game elements, users walk around the park to complete missions. Upon completion of a mission, the device sends the information to a server, and points are added to the user's account. These points can be converted into real-world assets through an electronic payment system once certain conditions are met.

[0625] As a concrete example, on a holiday afternoon, the server generates a "nature exploration mission" based on sunny weather information. Users explore the park using their devices, finding virtual plants and animals displayed in augmented reality and collecting points. The collected points are later provided to the user as a reward through an electronic payment system. In this way, active interpersonal interaction in the park and the revitalization of the local community are promoted.

[0626] The following describes the processing flow.

[0627] Step 1:

[0628] The server collects park map data, current weather, time of day, and participant profile information. Based on this, it uses an AI algorithm to generate game scenarios and missions suitable for multiple participants.

[0629] Step 2:

[0630] The server distributes the generated game data to the participants' devices. This data includes information on the placement of AR objects, mission details, and methods for interaction between participants.

[0631] Step 3:

[0632] The device uses its camera to overlay AR objects onto the surrounding scenery based on game data received from the server. A screen is then prepared that allows the user to start the game.

[0633] Step 4:

[0634] Users walk around the park with their devices, finding and interacting with mission objects displayed in augmented reality. User actions (such as taps and swipes) are detected by the device in real time.

[0635] Step 5:

[0636] The device acquires the user's current location information via GPS and dynamically adjusts the game content according to the user's movements. This provides the user with an appropriate gaming experience.

[0637] Step 6:

[0638] When a user completes a mission, the device sends the completion status to the server. Based on this information, the server adds points to the user's account.

[0639] Step 7:

[0640] The server periodically converts the accumulated points into actual reward points via an electronic payment system and reflects them in the user's account.

[0641] Step 8:

[0642] Users can check their point status through the app and redeem their earned points for use in other services. This concludes the game experience in the park.

[0643] (Example 1)

[0644] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] Conventional augmented reality virtual experience systems have struggled to generate game content that responds to environmental changes and participants' dynamic behavior. Furthermore, they lacked sufficient interaction to encourage cooperation and competition among users, and the process of providing game results as real-world rewards was complex. This invention aims to solve these problems and provide a more engaging and interactive virtual experience.

[0646] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0647] In this invention, the server includes means for generating game content using a generation AI model based on environmental information, means for distributing the generated game content to multiple electronic devices in real time, and means for displaying virtual objects superimposed on a real-world landscape based on user movement information acquired via the electronic devices. This makes it possible to provide game content that flexibly responds to environmental changes and the dynamic behavior of users, promote cooperation and competition among interactive participants, and immediately provide game results as real-world rewards.

[0648] A "park" is a public facility equipped with a natural environment, playground equipment, and walking paths, where people gather for recreation and relaxation.

[0649] An "electronic device" is a device that has the ability to process and display information, and specifically refers to smartphones, tablets, and the like.

[0650] Augmented reality technology is a technology that overlays computer-generated visual information onto the real environment, integrating reality and digital information.

[0651] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[0652] A "virtual experience" is an experience obtained by immersing participants in a digital environment separate from the real world.

[0653] "Environmental information" refers to data about the physical location and its surrounding conditions, including information such as weather, topography, and time of day.

[0654] "Game content" refers to the plan and instructions for an activity that consists of the objectives, goals, and procedures that participants will perform.

[0655] "Real-time" refers to the instantaneous processing or response that occurs the moment an operation or event takes place.

[0656] A "virtual object" is a non-physical object created by a computer and displayed through digital space or augmented reality technology.

[0657] "Movement information" refers to a collection of data such as the position, direction, and speed of a specific individual or object as it moves.

[0658] A "reward" is a monetary, physical, or digital benefit given for a particular action or achievement.

[0659] An "electronic payment system" is an infrastructure for exchanging money in a digital format via the internet.

[0660] Three main elements are involved in implementing this invention: the server, the terminal, and the user.

[0661] The server collects information from external databases and sensors to handle the park's topography and environmental information. Specifically, it uses map APIs and weather information APIs, and generates game content using a generative AI model based on this information. The generative AI model is implemented using machine learning frameworks such as TensorFlow and automatically generates new missions. Specific examples of generated content include tasks such as "On a sunny day, find and record five types of virtual birds in the park." This generated content is sent to the terminal in real time using protocols such as WebSocket.

[0662] A device is an electronic device owned by the user, typically a smartphone or tablet. This device has an application using augmented reality technology installed, developed using tools such as Unity. Based on game data received from a server, this application overlays virtual objects onto the real-world scenery via the device's camera. In addition, it uses the device's built-in sensors (GPS and gyroscope) to detect the user's location and movements, analyzes the data obtained, and updates the game content as needed.

[0663] Users move around the park with their devices and participate in the game through virtual objects and missions displayed in augmented reality. For example, they might receive a prompt such as "Find five different virtual birds in the park and earn a total of 100 points or more," and aim to achieve this goal. Upon completing a mission, the device reports the progress to the server, and points are added to the user's score based on their game performance. These points are converted and provided to the user as valuable rewards through an electronic payment system.

[0664] As a result, this system provides a flexible gaming experience that adapts to the environment, encourages users to utilize the park, and enhances interaction among participants.

[0665] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0666] Step 1:

[0667] The server collects topographic and environmental data for the park. This input is obtained by utilizing map APIs and weather APIs to acquire the latest data in real time. Data processing includes analyzing information obtained from various sensors and converting it into the required format. As output, this information is stored in an internal database for use in the next step.

[0668] Step 2:

[0669] The server generates game content using a generative AI model based on collected terrain data and environmental information. The AI ​​model receives information about the weather, time of day, and location, and processes this data to generate specific game missions. For example, a mission such as "Find a specific virtual object at a specific location within the park" might be created and output.

[0670] Step 3:

[0671] The server delivers the generated game content to each user's device in real time. Based on the output of the generation AI model, the game data is rapidly delivered using the WebSocket protocol. As output, the game's instructions and objectives are displayed on the user's device.

[0672] Step 4:

[0673] The device overlays virtual objects onto the real-world scenery based on game data received from the server. It activates the camera and uses the obtained visual data to render 3D objects. The input consists of camera footage and data from the server, and the output is the AR view on the user's display.

[0674] Step 5:

[0675] The device uses cameras and sensors to detect the user's location and movements in real time. Data from GPS, accelerometer, and gyroscope sensors is input and analyzed to track the user's movements. Based on the detected information, the device dynamically updates the game progress and generates output that provides feedback to the user.

[0676] Step 6:

[0677] Users explore the park according to the game's instructions using the device's AR view and progress through missions. User actions are captured as input from the device, and mission completion and progress are evaluated. As output, the mission completion status is displayed on the screen, and guidance is provided to lead to the next action.

[0678] Step 7:

[0679] The device sends information to the server when the user completes a mission. The collected data (completion information) is treated as input, and points are calculated based on this. As output, points are added to the user's account, and this information is stored on the server for subsequent reward processes.

[0680] Step 8:

[0681] The server verifies that the accumulated points meet certain conditions and provides the reward via the electronic payment system. The calculated points are entered and converted into actual rewards for the user through the electronic payment system. The output is the transmission of the reward to the user's electronic payment account.

[0682] (Application Example 1)

[0683] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0684] There is a need for new methods to improve the consumer experience at facilities that provide consumer goods and to increase purchasing intent within those facilities. Furthermore, there is a significant lack of effective means to attract participants' interest and encourage sustainable use of these facilities. Therefore, the challenge lies in providing activities that consumers can enjoy participating in within the facilities, along with corresponding reward systems.

[0685] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0686] In this invention, the server includes means for collecting participation information from users in the facility and generating game content based on that information, means for distributing the generated game content to multiple communication devices, and means for dynamically generating game scenes based on the location information of users acquired via the communication devices. As a result, consumers can deepen their interest in products and services while having fun through games utilizing augmented reality technology within the facility, and effectively use points as rewards for commercial services.

[0687] A "facilities that provide consumer goods" refers to a physical or virtual location that directly provides goods or services to consumers.

[0688] "Communication devices" refer to electronic devices capable of sending and receiving information, including smartphones and tablets.

[0689] Augmented reality technology is a technique that overlays computer-generated information onto the real world, enabling users to interact with the real world and virtual objects.

[0690] "Participant information" refers to information about participants in a game or event, including their name, location, and behavioral patterns.

[0691] "Generating game content" means constructing playable game scenarios and tasks based on participant information and the environment at the time.

[0692] "Distributing to a communication device" means transmitting generated digital content to a designated device via the internet or other communication network.

[0693] "Location information" refers to data on physical or virtual locations identified using GPS or other location tracking technologies.

[0694] "Dynamically generating game scenes" means changing and adapting game settings and scenarios in real time according to the player's position and actions.

[0695] "Commercial service perks" refer to rewards or special treatment that customers receive when purchasing goods or services, such as discounts or bonus items.

[0696] The system for realizing this invention is designed to provide interactive games using augmented reality technology within facilities that sell consumer goods. At the core of this system is a server, which is responsible for various data processing tasks.

[0697] The server collects participation information from consumers playing games within the facility using communication devices (e.g., smartphones). This includes user location information, behavioral patterns, and areas of interest. Based on this information, the server uses a generative AI model to generate appropriate game content in real time. The generated game content is then delivered to the participants' communication devices via AR development frameworks such as ARCore and ARKit.

[0698] The device uses cameras and sensors to acquire the user's location information and accurately display AR content within the facility. Users can search for challenges generated via communication devices and compete with other participants. Points earned through the game can be converted into rewards for commercial services, encouraging users to make purchases.

[0699] As a concrete example, scanning a specific product within a facility triggers an AR game based on that product. This game progresses as the user solves problems related to the product. By accumulating and exchanging points, users can obtain specific discounts and benefits.

[0700] Examples of prompts for a generative AI model include:

[0701] "Create an AR mini-game based on a specific product scanned by a smartphone. Include interesting information and challenges to capture the user's interest."

[0702] The requirements are communicated to the server in this manner.

[0703] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0704] Step 1:

[0705] The server receives location and behavioral data transmitted from participants within the facility via communication devices as input. It processes this data to understand the current context of the participating users. Through this process, the server obtains foundational data for designing game scenarios tailored to each user.

[0706] Step 2:

[0707] The server sends a prompt to the generating AI model based on the collected data, and generates specific game content. This prompt is "Generate an AR mini-game based on a specific product." The generating AI model then operates and outputs data including the game's challenges and storyline.

[0708] Step 3:

[0709] The server uses the game content obtained from the generated AI model to construct game data using an AR development framework (ARCore or ARKit). This game data is processed to include the positions of virtual objects and details of user interactions. The constructed game data is then distributed to each user's communication device.

[0710] Step 4:

[0711] The device uses game data received from the server to overlay AR content onto the camera feed. The device tracks the user's movements and location in real time and appropriately updates the AR content as the user moves within the facility. In this process, the device performs data calculations based on the user's actions and adjusts the screen display accordingly.

[0712] Step 5:

[0713] Users participate in an AR game displayed on their device and work to complete designated tasks. Information about when a user completes a mission is recorded on the device and sent to the server. As a result, users earn points, which they can later use as rewards for commercial services.

[0714] Step 6:

[0715] The server calculates points based on the received mission completion information and adds them to the user's account. If the points meet certain conditions, they are converted into a form that can be used as a reward for commercial services. The results of this process are reflected in the user's electronic trading account according to their usage.

[0716] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0717] This invention provides a system for multiple users to enjoy games using augmented reality technology with information terminal devices in a park. By further incorporating an emotion engine, this system can recognize the emotional state of each user and individually optimize the game experience.

[0718] The server collects a map of the park, participant profiles, and location information, and uses AI to generate a game scenario. This game data is then distributed to the participants' devices. The game data includes the placement of AR objects, missions, and interaction methods.

[0719] The terminals are devices such as smartphones carried by participants, and they use cameras to collect user facial expression data. This facial expression data is analyzed by an emotion engine within the terminal to identify the user's emotional state. The analysis results are sent to a server, which uses this information to adjust the game content and difficulty. For example, if the server determines that the user is bored, it will either increase the pace of the game or add a new challenge.

[0720] Meanwhile, users use their devices to explore the park, finding objects displayed in augmented reality to complete missions. Once a mission is completed, the device sends information, along with emotional data, to the server, and points are added to the user's account. These points can then be converted into something of value through an electronic payment platform.

[0721] As a concrete example, the server generates a game called "Teamwork Challenge" in a sunny park. Users move around the park with their devices and cooperate with other participants to complete missions. The emotion engine detects whether the user is feeling stressed and, if so, displays helpful hints on the device to facilitate communication with other team members. This makes the game experience more immersive and naturally promotes interaction between users.

[0722] The following describes the processing flow.

[0723] Step 1:

[0724] The server references a database containing participant registration information and uses an AI algorithm to generate a game scenario suitable for each participant based on the current conditions of the park (e.g., weather, time of day, number of participants).

[0725] Step 2:

[0726] The server sends the generated game data to each participant's device. This data includes the location of AR objects, mission details, and expected interaction details.

[0727] Step 3:

[0728] The device analyzes game data received from the server, activates the camera, and overlays AR objects onto the actual park scenery. This allows the user to start the game.

[0729] Step 4:

[0730] The device uses its camera to capture the user's facial expressions and analyzes that data in real time using its built-in emotion engine. The emotion engine identifies the user's emotional state.

[0731] Step 5:

[0732] Users move around the park with their devices, finding objects displayed in augmented reality through interaction and completing game missions.

[0733] Step 6:

[0734] The device transmits the user's location and emotional state information to the server. Based on this information, the server dynamically adjusts the game content, adaptively changing the game's difficulty and scenario progression.

[0735] Step 7:

[0736] When a user completes a mission, the device reports the result to the server. The server adds points to the user's account and converts the reward into an electronic payment system according to the user's achievement level.

[0737] Step 8:

[0738] The server uses the collected sentiment data to send information to the user's device that enhances in-game cooperation and competition to make the game more enjoyable. The device then displays this information as an interface for the user.

[0739] (Example 2)

[0740] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0741] The present invention aims to provide a system that, when offering interactive games using augmented reality technology in a park, can reflect the emotional state of individual users in real time, thereby making the game experience more personalized and optimized. It also aims to promote natural cooperation and competition among users and enhance motivation for playing the game.

[0742] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0743] In this invention, the server includes means for collecting geographic and profile information of participants and generating game scenarios using a generative AI model based on that information; means for analyzing user facial expression data acquired by the cameras of each information terminal device and identifying the user's emotional state using an emotion engine; and means for adjusting the game content and difficulty in real time based on the identified emotional state. This makes it possible to provide a game experience optimized for each user and to effectively promote cooperation and competition among users.

[0744] An "information terminal device" is a portable device that a user can carry, has functions to utilize augmented reality technology, and can collect the user's location information and emotional state.

[0745] Augmented reality technology is a technique that overlays digital data and images onto real-world images and information, allowing users to experience virtual information realistically through their vision.

[0746] A "generative AI model" is a form of artificial intelligence that can generate new game scenarios and content based on collected data.

[0747] An "emotion engine" is software or an algorithm that analyzes facial expression data acquired from a device to identify the user's emotional state.

[0748] An "electronic payment platform" is a system that converts points and other digital currencies into real-world value, enabling transactions and exchanges.

[0749] A "prompt" is a sentence containing a question or instruction that is input into a generative AI model, and is used to elicit specific answers or scenarios.

[0750] "Geographic information" refers to data that indicates the physical location of a user or device, and is obtained through GPS or other location detection technologies.

[0751] This invention is a system for multiple users in a park to enjoy games utilizing augmented reality technology using information terminal devices. The server collects users' geographical and profile information and generates new game scenarios using a generative AI model based on this information. In this process, the server gives instructions to the generative AI model using prompt statements. An example of a prompt statement is, "Design a scenario that is active and enhances teamwork."

[0752] The terminals are smartphones, tablets, and other devices owned by participating users, and they use their cameras to capture facial expression data. The captured data is analyzed by an emotion engine within the terminal to determine the user's emotional state. This analysis result is sent to a server and used to adjust the game content and difficulty in real time.

[0753] Users use their devices to freely move around the park, discovering digital objects through augmented reality (AR) technology and completing various missions. This allows users to enjoy a new experience where real space and digital information merge. Furthermore, each time a mission is completed, the device sends the information to a server, and points are added to the user's account. These points can be converted into tangible value through an electronic payment platform and exchanged for digital goods and services.

[0754] In this way, the invention combines augmented reality technology and generative AI models to provide personalized gaming experiences and enable dynamic game progression that responds to the user's emotional state.

[0755] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0756] Step 1:

[0757] The server collects geographical information within the park and user profile data. This includes GPS data and profile information previously entered by the user. Based on the entered geographical and profile information, the server determines the user's current location and prepares the basic data for formulating appropriate scenarios within the game context. The output is a dataset based on location and user attribute information.

[0758] Step 2:

[0759] The server inputs the dataset obtained in Step 1 into the AI ​​model and generates a new game scenario using prompt statements. These prompt statements instruct the AI ​​in the form of "Create AR missions suitable for each area of ​​the park." The AI ​​model processes the data and outputs a game scenario that includes the game's storyline, tasks, and AR object placement information.

[0760] Step 3:

[0761] The server delivers the generated game scenario to each user's device. The device receives the scenario information, builds a visual interface for the user, and prompts them to execute AR objects and tasks. Based on the input game scenario, it dynamically generates interactive elements to be displayed on the device. As output, it provides the user with a playable game environment in their field of view.

[0762] Step 4:

[0763] The device uses the user's camera to collect facial expression data. This data is analyzed by an emotion engine to identify the user's emotional state (e.g., happy, bored, excited). The output derived from the analyzed data is real-time feedback on the user's emotions.

[0764] Step 5:

[0765] Based on the sentiment analysis results from step 4, the server adjusts the specific game content and difficulty in real time. If it determines that the user is bored, it increases the pace of the game or offers additional challenges. This process takes sentiment data as input and generates output to optimize the existing game settings.

[0766] Step 6:

[0767] When a user completes an AR mission, the device sends its completion status to the server and simultaneously updates the user's emotional state. Based on the completion data as input, the server adds points to the user's account. The output is an increase in points, and this information is immediately fed back to the user.

[0768] Step 7:

[0769] The server converts points added to a user's account into tangible value through an electronic payment platform. Based on the input data (point information), it determines the corresponding reward or service and transfers the actual value to the user's electronic wallet or payment account as output.

[0770] (Application Example 2)

[0771] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0772] There is a growing need to provide individually optimized user experiences using advanced technology in real-world environments such as moving objects and parks. However, existing systems lack sufficient dynamic adjustments that utilize user emotions and location information, resulting in a failure to enhance user satisfaction. Furthermore, there are limited ways to effectively convert achieved scores into value.

[0773] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0774] In this invention, the server includes means for collecting participation information from people in parks or on moving objects and generating scene content based on that information; means for distributing the generated scene content to multiple information processing devices; and means for analyzing emotional states based on facial expression information of people acquired through the information processing devices and dynamically adjusting the content. This makes it possible to provide an experience optimized according to the user's emotions and to convert achievement scores into effective value in real environments such as moving objects and parks.

[0775] "Parks and mobile spaces" refer to spaces where people can gather and engage in activities, or spaces where users can gain benefits through physical experiences, such as self-driving vehicles.

[0776] An "information processing device" is an electronic device that has the function of manipulating and processing digital data, and includes smartphones, tablets, and in-car computers.

[0777] Augmented reality technology refers to the technology that enhances and modifies the real world environment using digital technology, and is also known as AR.

[0778] "Means for generating scene content" refer to technologies and algorithms used to create appropriate scenarios and experiences based on user information.

[0779] "Means of distribution" refers to the process of transferring generated digital content to various information processing devices using the internet or other means.

[0780] "Methods for analyzing emotional states based on facial expression information" refers to technologies that use facial feature data acquired by cameras and sensors to analyze a user's emotions.

[0781] "Means of dynamically adjusting content" refers to technologies that change the information and experience provided in real time according to the user's state and environment.

[0782] "Methods for converting scores into points for an electronic payment system" refers to methods of converting points earned in games or other activities into digital currency or point systems, making them available to users as economic value.

[0783] In one embodiment of this invention, the system provides an individually optimized user experience by utilizing information processing devices in parks and mobile vehicles. In particular, the system analyzes the user's emotions using augmented reality technology and an emotion engine, and dynamically adjusts the content based on that information.

[0784] The program is developed using programming languages ​​such as Python and JavaScript. High-performance computers or cloud services can be used for the server. The server collects participant location information and facial expression data, and generates and adjusts games and scenarios using a generative AI model. TensorFlow and PyTorch may be used for the AI ​​model.

[0785] The terminal is a smartphone or a device equipped with hardware sensors, and it uses OpenCV to detect the user's face from images acquired by the camera. The face data is sent to the server in real time, and the emotional state is analyzed. Based on the analysis results, the server adjusts and delivers the game scenario and experience content.

[0786] For example, if a user is relaxing in an autonomous vehicle, the server will select calming music in real time and provide relaxing AR content for the passenger. Furthermore, the suggested music and content can change according to the user's emotional state.

[0787] As an example of a prompt, instructions such as, "The passengers appear relaxed based on their facial expressions. Please suggest music and AR content that suits them," are input to the generating AI model, and the system then recommends and provides the most suitable content.

[0788] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0789] Step 1:

[0790] The server collects participation information from people in parks and on moving objects and stores it in a database. It takes location information and profile data of individuals as input and stores it in the database. This prepares the basic information necessary for subsequent processing steps.

[0791] Step 2:

[0792] The device activates its camera and captures the user's face to acquire facial expression data. Using the real-time captured image data as input, OpenCV is used to detect and analyze the face. This identifies the user's emotional state and generates analysis results.

[0793] Step 3:

[0794] The terminal sends the analysis results to the server. The analyzed emotion data is used as input and sent to the server. Based on the received emotion data, the server utilizes an AI model to generate appropriate game scenarios and experience content. The adjusted game data is then generated as output.

[0795] Step 4:

[0796] The server distributes the generated game data to the terminal. It uses the generated game scenario and experience content as input and delivers it to the terminal. This allows the user to experience content that is adjusted to their emotional state.

[0797] Step 5:

[0798] Users participate in the experience using their devices, and the devices collect results and new score data. The user's actions and achievements are captured as input and fed back to the server. Based on this information, the server generates scores and converts them into electronic payment points.

[0799] Step 6:

[0800] The server transfers the converted points to the user's electronic payment account. Using points that meet certain conditions as input, the system sends the points to the account via the electronic payment system. This allows the user to use the earned points as actual value.

[0801] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0802] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0803] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0804] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0805] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0806] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0807] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0808] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0809] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0810] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0811] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0812] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0813] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0814] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0815] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0816] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0817] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0818] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0819] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0820] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0821] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0822] The following is further disclosed regarding the embodiments described above.

[0823] (Claim 1)

[0824] A system that provides games using augmented reality technology using information terminal devices used by multiple users in a park,

[0825] A means of collecting participation information from users in the park and generating game content based on that information,

[0826] A means for distributing the generated game content to multiple information terminal devices,

[0827] A means for dynamically generating game scenes based on the user's location information acquired via the aforementioned information terminal device,

[0828] A means of converting points accumulated by users according to completed game missions into points for an electronic payment system,

[0829] A system that includes this.

[0830] (Claim 2)

[0831] The system according to claim 1, which provides an interface that encourages users to cooperate or compete with each other based on their progress in the game.

[0832] (Claim 3)

[0833] The system according to claim 1, wherein when the aforementioned points meet certain conditions, it performs a process of transferring the points to the user's electronic payment account.

[0834] "Example 1"

[0835] (Claim 1)

[0836] A system that provides a virtual experience using augmented reality technology with electronic devices used by multiple users in a park,

[0837] A means of generating game content using an AI model based on environmental information,

[0838] A means of distributing the generated game content to multiple electronic devices in real time,

[0839] A means for displaying a virtual object superimposed on a real-world landscape based on user movement information acquired via the aforementioned electronic device,

[0840] Based on collected user activity data, a means of analyzing mission progress and providing performance-based rewards through an electronic payment system,

[0841] A system that includes this.

[0842] (Claim 2)

[0843] The system according to claim 1, which provides a procedure that encourages users to cooperate or compete with each other based on the progress of the aforementioned mission.

[0844] (Claim 3)

[0845] The system according to claim 1, which, when the aforementioned reward meets certain conditions, processes a process to send said reward to the user's electronic payment account.

[0846] "Application Example 1"

[0847] (Claim 1)

[0848] A system that provides games using augmented reality technology using communication devices used by multiple users in a facility that provides consumer goods,

[0849] A means of collecting participation information from users at the facility and generating game content based on that information,

[0850] A means for distributing the generated game content to multiple communication devices,

[0851] A means for dynamically generating a game scene based on the user's location information acquired via the aforementioned communication device,

[0852] A means of converting points accumulated by users according to completed game missions into rewards for commercial services,

[0853] A system that includes this.

[0854] (Claim 2)

[0855] The system according to claim 1, which provides an interface that encourages users to cooperate or compete with each other based on their progress in the game.

[0856] (Claim 3)

[0857] The system according to claim 1, which, when the aforementioned points meet certain conditions, executes a process to transfer the points to the user's electronic trading account.

[0858] "Example 2 of combining an emotion engine"

[0859] (Claim 1)

[0860] A system that provides a game using augmented reality technology using information terminal devices used by multiple users in a park,

[0861] A means of collecting geographical and profile information of participants and generating game scenarios using a generative AI model based on that information,

[0862] A means for distributing the generated game content to multiple information terminal devices,

[0863] A means of analyzing user facial expression data acquired by the cameras of each information terminal device and using an emotion engine to identify the user's emotional state,

[0864] A means of adjusting the game content and difficulty in real time based on identified emotional states,

[0865] A means of converting points accumulated according to game progress into points for an electronic payment platform,

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1, which dynamically adjusts the interface by utilizing the emotional state of users during a game in order to promote cooperation and competition among users.

[0869] (Claim 3)

[0870] The system according to claim 1, wherein when the aforementioned points meet certain conditions, it performs a process of transferring the points to the user's electronic payment account.

[0871] "Application example 2 when combining with an emotional engine"

[0872] (Claim 1)

[0873] A system that uses information processing devices used by multiple people in parks and on mobile devices to provide games and a comfortable environment using augmented reality technology,

[0874] A means of collecting participation information from people in parks and on moving objects, and generating scene content based on that information,

[0875] A means for distributing the generated scene content to multiple information processing devices,

[0876] A means for analyzing a person's emotional state and dynamically adjusting the content based on facial expression information acquired via the aforementioned information processing device,

[0877] A means of converting points accumulated based on game missions and emotional states achieved by players into points for an electronic payment system,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, which provides an interface that encourages cooperation or competition among people based on the achievement status of a scene.

[0881] (Claim 3)

[0882] The system according to claim 1, wherein if the score meets certain conditions, it performs a process of transferring the score to a person's electronic payment account. [Explanation of Symbols]

[0883] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A system that provides games using augmented reality technology using information terminal devices used by multiple users in a park, A means of collecting participation information from users in the park and generating game content based on that information, A means for distributing the generated game content to multiple information terminal devices, A means for dynamically generating game scenes based on the user's location information acquired via the aforementioned information terminal device, A means of converting points accumulated by users according to completed game missions into points for an electronic payment system, A system that includes this.

2. The system according to claim 1, which provides an interface that encourages users to cooperate or compete with each other based on their progress in the game.

3. The system according to claim 1, wherein when the aforementioned points meet certain conditions, it performs a process of transferring the points to the user's electronic payment account.

Images