System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving and analyzing environmental data in a user's home; means for generating a ghost or horror scenario suitable for the user's home based on an analysis result; means for transmitting the generated ghost and scenario information to a terminal; means for recognizing a user's home environment and user's behavior in the terminal and displaying the ghost at an appropriate timing and place; means for monitoring and collecting user's reaction data; and means for analyzing the collected reaction data and dynamically updating the horror scenario.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional horror games and haunted houses mainly use fixed scenarios and presentations, and are not optimized for each user's living environment or behavior, making it impossible to provide a personalized horror experience. The present invention aims to provide a more realistic and immersive horror experience by using ghosts and scenarios dynamically generated based on the user's living environment to realize an individually optimized interactive horror experience. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a means for receiving and analyzing environmental data of a user's residence, a means for generating ghosts and a horror scenario suited to the user's residence based on the analysis results, a means for transmitting the generated ghost and scenario information to a terminal, a means for recognizing the user's residence environment and user behavior on the terminal and displaying ghosts at appropriate times and locations, a means for monitoring and collecting user reaction data, and a means for analyzing the collected reaction data and dynamically updating the horror scenario. Specifically, the present invention includes a means for analyzing the furniture arrangement and floor plan of the user's residence and determining the location where the ghost will appear and the content of the presentation based on this, and a means for monitoring the user's voice reactions to detect when the user felt particularly scared and updating the horror scenario based on this information.

[0006] "Environmental data" refers to information such as the room layout, furniture arrangement, and lighting conditions in the user's residence.

[0007] "Analysis" refers to the process of examining and evaluating collected environmental data and user response data in detail to extract useful information.

[0008] A "ghost" is a virtual entity created to frighten the user, appearing at specific locations and times within a horror scenario.

[0009] A "horror scenario" refers to a series of events or performances designed to provide a frightening experience to the user.

[0010] "Terminal" refers to an information processing device such as a smartphone or tablet that a user owns.

[0011] "Recognition" refers to the device using cameras and sensors to grasp the user's living environment and behavior in real time.

[0012] "Timing" refers to the right moment for a ghost to appear or the right point in time for a scenario to unfold.

[0013] "Location" refers to a specific location within the user's residence where a ghost appears.

[0014] "Reaction data" refers to data that represents a user's behavior and emotional reactions, and is collected based on audio, camera footage, etc.

[0015] "Dynamic updating" refers to the process of changing the scenario in real time based on collected user reaction data to provide an optimized horror experience.

[0016] "Furniture layout" refers to information indicating the location of each piece of furniture in the user's residence.

[0017] "Floor plan" refers to information that indicates the layout and shape of each room and space in the user's residence.

[0018] "Voice response" refers to voice data such as voices or screams emitted when a user feels fear.

[0019] "Detection" refers to the process of recognizing and identifying a particular response or behavior. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention provides an interactive horror experience by utilizing environmental data of the user's home. The program and processing of this system will be described in detail below.

[0042] Server Operation

[0043] Receiving and analyzing environmental data

[0044] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[0045] Generating ghosts and horror scenarios

[0046] The server generates ghost and horror scenarios that best fit the user's environment based on the analyzed residential information. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[0047] Sending scenario information

[0048] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[0049] Device behavior

[0050] environmental awareness

[0051] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[0052] Displaying ghosts and running scenarios

[0053] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[0054] User behavior monitoring

[0055] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the device records the voice and movements and sends them to the server. This data is important for showing when the user felt particularly scared.

[0056] Feedback and scenario updates

[0057] Analysis of user responses and scenario updates

[0058] The server analyzes the user's reaction data sent from the device and identifies the elements that frighten the user. For example, if the user is particularly afraid of the dark, the next ghost displayed will be set to appear from a darker location. In this way, the scenario is dynamically updated, aiming to maximize the sense of fear.

[0059] Providing a continuous horror experience

[0060] The updated scenario information is then sent back to the device, which then receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[0061] Specific examples

[0062] For example, let's say a user starts an experience in their living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghostly hand at the appropriate time. If the user jumps in surprise at this effect, the video and audio of the user's reaction are recorded and sent to the server. The server adjusts the next scenario based on this reaction, continuously evolving the terrifying experience.

[0063] As described above, the present invention is a system that dynamically links with the user's home environment to provide a realistic and immersive horror experience.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[0067] Step 2:

[0068] Device: Sends uploaded photos and floor plan data to the server.

[0069] Step 3:

[0070] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[0071] Step 4:

[0072] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[0073] Step 5:

[0074] Server: Sends the generated ghosts and scenario information to the user's device.

[0075] Step 6:

[0076] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[0077] Step 7:

[0078] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[0079] Step 8:

[0080] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or expresses surprise.

[0081] Step 9:

[0082] Terminal: Sends user reaction data to the server.

[0083] Step 10:

[0084] Server: Analyzes the user's reaction data and identifies the elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[0085] Step 11:

[0086] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost may appear from a darker location.

[0087] Step 12:

[0088] Server: Sends updated scenario information to the device.

[0089] Step 13:

[0090] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[0091] In this way, the present invention provides a dynamic and individually optimized horror experience based on the user's living environment.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Conventional horror experience systems have had the problem of being unable to provide an immersive experience, as it is difficult to provide a frightening experience that corresponds to the user's individual living environment and behavior. Furthermore, it is difficult to analyze the user's reactions in real time and dynamically update the scenario. This creates the problem of users becoming bored.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes means for receiving and analyzing environmental information about the user's residence, means for generating a virtual image and a horror experience scenario suited to the user's residence based on the analysis results, and means for transmitting the generated virtual image and scenario information to the information terminal. This makes it possible to provide a horror experience suited to the user's individual environment. The information terminal also includes means for detecting the user's residence environment and user behavior and displaying a virtual image at an appropriate time and place, means for monitoring and collecting user reaction data, and means for analyzing the collected reaction data and dynamically updating the horror experience scenario. This makes it possible to individually optimize the scenario based on the user's reaction and provide a sustained and immersive horror experience.

[0097] 1. "User's place of residence" refers to the place where the user lives on a daily basis, including the environment such as a house or apartment.

[0098] 2. "Environmental information" refers to information about the user's place of residence, including data such as room layout, furniture arrangement, brightness, temperature, and sound.

[0099] 3. "Virtual images" refer to images or visual effects that do not exist in reality but are visually displayed, including horror effects such as ghosts.

[0100] 4. "Horror experience scenario" refers to a series of actions or storylines designed to instill fear in the user.

[0101] 5. "Information terminal" refers to a device used by a user, including a smartphone, tablet, computer, etc.

[0102] 6. “Analysis” means the process or method of examining collected data in detail and extracting necessary information.

[0103] 7. "Detection" refers to sensing the environment or user behavior using devices such as sensors and cameras.

[0104] 8. "Monitoring" means continuously observing user behavior and reactions and obtaining necessary data.

[0105] 9. "Collection" refers to the compilation and retention of data obtained through surveillance.

[0106] 10. "Dynamic updates" refers to changing the scenario or presentation in real time according to the situation.

[0107] 11. "Individual optimization" means providing optimal content tailored to the characteristics of each user based on collected data.

[0108] 12. “Generative AI Model” means a computational model that uses artificial intelligence techniques to generate new data or content.

[0109] The present invention is a system that provides an interactive horror experience by utilizing environmental information of a user's residence. This system realizes an immersive horror experience in real time using a terminal, a server, and cameras and sensors for monitoring the user's actions and reactions. Specific embodiments of this system are described below.

[0110] Server Operation

[0111] Receiving and analyzing environmental information

[0112] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app. After receiving the data, the server analyzes this data to determine the room layout and furniture placement information. For example, if a sofa or bookshelf is located in the living room, it will identify their locations. Image recognition algorithms are used for the analysis.

[0113] Generating ghosts and horror scenarios

[0114] Based on the analyzed residential information, the server generates virtual images and horror scenarios that are optimal for the user's environment. The generative AI model is used to devise scenarios and determine the timing and content of the scenes. For example, it can generate scenes in which a ghostly hand reaches out from under a sofa or a closet door opens on its own.

[0115] Sending scenario information

[0116] The generated virtual images and horror scenarios are sent from the server to the information terminal, where data is communicated and synchronized in real time to ensure a smooth user experience.

[0117] Device behavior

[0118] environmental awareness

[0119] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the room's brightness, furniture layout, and the user's location, and sends the information to the server. For example, the device may detect that the user is near a bookshelf.

[0120] Displaying ghosts and running scenarios

[0121] Based on the scenario information sent from the server, the device displays a virtual image superimposed on the user's real environment. For example, if the user is sitting on a sofa, a ghost's hand will appear to reach out from under the sofa.

[0122] User behavior monitoring

[0123] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the voice and movement are recorded and sent to the server. This allows the device to understand when the user is feeling particularly scared.

[0124] Feedback and scenario updates

[0125] Analysis of user responses and scenario updates

[0126] The server analyzes the user's reaction data sent from the device and identifies the elements of fear that should be enhanced in the next scenario. For example, if the user has a strong fear of the dark, the server will set the next scenario to have ghosts appear from darker places.

[0127] Providing a continuous horror experience

[0128] The server then sends the updated scenario information back to the device, which then receives it and provides a new horror experience, ensuring that users always receive a fresh, individually optimized horror experience.

[0129] Specific examples

[0130] 1. The user begins the experience in their living room

[0131] The server analyzes photos of the living room and identifies the location of the bookshelf and sofa.

[0132] Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device.

[0133] 2. Device-based environment recognition and scenario execution

[0134] The device uses cameras and sensors to scan the user's living environment and detects when the user approaches the sofa.

[0135] A ghostly hand appears at the appropriate time to create a frightening effect.

[0136] 3. User response and data transmission

[0137] When the user screams in surprise, their voice and movements are recorded by the terminal.

[0138] The terminal transmits this reaction data to the server.

[0139] 4. Feedback analysis and scenario update

[0140] The server analyzes the user's reaction data and identifies areas that need to be strengthened in the next scenario. For example, if darkness causes fear, the next performance could have a virtual image appear from a dark place.

[0141] 5. Submit new scenarios and continue the experience

[0142] The server then sends the updated scenario back to the device, which receives it and provides a new horror experience.

[0143] Prompt Sentence Examples

[0144] "Generate a scenario where something crawls out from under the sofa the moment the user enters the living room. Think of a realistic effect that will surprise the user."

[0145] "Create a scenario where a ghost appears from the shadow of the bookshelf when the user goes to get a book. Set the timing to make the user feel scared."

[0146] "I thought of an eerie sound playing in the dark, and when the user heads in the direction of the sound, a ghost will appear."

[0147] As described above, the present invention dynamically links with the user's home environment to provide an individually optimized horror experience.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Program processing flow

[0150] Server Operation

[0151] Step 1:

[0152] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app.

[0153] Input: Photo and floor plan data of the residence submitted by the user.

[0154] Output: Received residence location data.

[0155] Specific operation: Use the file receiving module to obtain image data from the user.

[0156] Step 2:

[0157] The server inputs the received data into an image recognition algorithm for analysis.

[0158] Input: Received residential location photo and floor plan data.

[0159] Output: Analyzed room layout and furniture placement information.

[0160] What it does: Uses image recognition software to identify furniture locations and room layouts.

[0161] Step 3:

[0162] The server uses a generative AI model to generate virtual images and horror experience scenarios based on the analysis results.

[0163] Input: Room layout and furniture placement information.

[0164] Output: Generated virtual images and frightening scenarios.

[0165] Specific operation: A prompt sentence is input into the generative AI model to generate a scenario. For example, it generates a scenario in which a ghost's hand reaches out from under the sofa.

[0166] Step 4:

[0167] The server transmits the generated virtual image and scenario information to the user's terminal.

[0168] Input: Generated virtual image and scenario information.

[0169] Output: Scenario data sent to the user's information terminal.

[0170] Specific operation: The scenario data is transmitted to the user terminal using the data transmission module.

[0171] Device behavior

[0172] Step 5:

[0173] The device uses cameras and sensors to scan the user's living environment in real time and transmits the data to a server.

[0174] Input: Real-world environment data acquired by cameras and sensors.

[0175] Output: The environment data sent to the server.

[0176] Specific operation: The environmental recognition module is used to detect the room brightness, furniture arrangement, and user position in real time.

[0177] Step 6:

[0178] Based on the scenario information sent from the server, the terminal displays a virtual image superimposed on the user's real environment.

[0179] Input: Scenario information sent from the server.

[0180] Output: Displayed virtual image and frightening experience scenario.

[0181] Specific operation: Using the AR (Augmented Reality) module, a ghost hand appears from under the sofa when the user approaches it.

[0182] Step 7:

[0183] The device uses a camera and microphone to monitor the user's actions and reactions in real time and transmits the data to a server.

[0184] Input: User behavior and voice data captured by camera and microphone.

[0185] Output: User response data sent to the server.

[0186] Specific actions: Use the monitoring module to record the user's voice and actions such as shouting or being surprised.

[0187] Feedback and scenario updates

[0188] Step 8:

[0189] The server analyzes the user's reaction data sent from the device and identifies the fear elements that should be strengthened in the next scenario.

[0190] Input: User reaction data sent from the device.

[0191] Output: Scenario with new and enhanced horror elements based on analysis.

[0192] Specific behavior: Use the data analysis module to identify the moments and elements that made users feel most scared.

[0193] Step 9:

[0194] The server generates new scenarios and sends them to the user's device, continually updating the terrifying experience.

[0195] Input: User's reaction data analysis results.

[0196] Output: New scenario data.

[0197] Specific operation: New conditions are input into the generative AI model to generate scenarios with enhanced fear elements, such as "a scenario in which a ghost appears from a dark place."

[0198] As described above, this system allows the server and device to cooperate to provide users with an individually optimized interactive horror experience.

[0199] (Application example 1)

[0200] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0201] Conventional horror experience systems do not adequately generate horror scenarios that are dependent on the user's real-world environment, nor do they dynamically update the scenarios based on the user's real-time actions and reactions. As a result, there are issues with the user's sense of fear and immersion being limited. In addition, it has been difficult to provide a horror experience through interaction in a virtual real space.

[0202] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0203] In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and a horror scenario appropriate for the user's residence based on the analysis results, means for transmitting the generated ghost and scenario information to the terminal, means for recognizing the user's residence environment and user behavior in the terminal and displaying a ghost at an appropriate time and place, means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for causing a ghost's hand to appear from a specific object when the user approaches the object in the virtual real space, and means for generating a darkness effect and causing a ghost to appear in that location when the user performs a certain action in a specific area in the virtual real space, thereby providing an interactive and immersive horror experience in both the user's real environment and the virtual real space.

[0204] "Environmental data in the user's residence" is data including the layout, furniture arrangement, lighting conditions, and other related information within the user's residence.

[0205] "Ghost and Horror Scenarios" are scenes of mysterious phenomena and terrifying experiences that are generated based on the user's interactions in their living environment and virtual real space.

[0206] "Terminals" are devices such as smartphones, smart glasses, and head-mounted displays that allow users to experience horror scenarios visually and aurally.

[0207] "Means for recognizing user behavior" refers to a function that detects information such as the user's position, movements, voice, and facial expressions in real time.

[0208] "User reaction data" refers to data including behavior, vocal responses, and physiological responses such as heart rate that a user exhibits in response to a horror scenario.

[0209] "Means for dynamically updating horror scenarios" is a function that analyzes collected user reaction data and changes the content and presentation of the scenario to provide the user with the optimal horror experience.

[0210] A "virtual real space" is a virtual reality environment, a digital space designed for users to experience.

[0211] The "effect of a ghostly hand appearing from the object" is an effect in which, when a user approaches a specific virtual object, a mysterious phenomenon (for example, a ghostly hand) visually occurs from the object.

[0212] "Means for creating a darkness effect and making a ghost appear in that location" refers to a production in which, when a user performs a specific action in a specific area of ​​the virtual real space, the lights in that area are dimmed and a ghost appears.

[0213] This invention is a system that utilizes data on a user's home environment to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[0214] The server has a means for receiving and analyzing environmental data of the user's residence, including photos and floor plan data of the user's residence. Based on this data, the server analyzes the room layout and furniture arrangement to generate specific ghost and horror scenarios.

[0215] Based on the analysis results, the server has the means to generate ghosts and horror scenarios that are appropriate for the user's residence. For example, if the user has a sofa in their living room, the server will generate a scenario in which a ghost's hand reaches out from under the sofa. This generated ghost and scenario information is sent to the device via a dedicated application.

[0216] The device has a means of recognizing the user's home environment and behavior. The device is equipped with a camera and sensors, and can detect the user's location and behavior in real time. For example, when the user approaches a specific location, that information is sent to the server.

[0217] The device also has a means to display ghosts at the appropriate time and place. Based on the scenario information sent from the server, the device can display ghosts superimposed on the user's real environment. For example, when the user is sitting on a sofa, the device will display an effect in which a ghost's hand reaches out from under the sofa.

[0218] Additionally, the device has a means of monitoring and collecting user reaction data. It uses a camera and microphone to record the user's actions and reactions and transmits the data to a server. This reaction data includes when the user shouts or shows surprise.

[0219] The server analyzes the collected reaction data and dynamically updates the horror scenario. It identifies the elements that frighten the user most and reflects this in the next scenario. For example, if the user is very scared of the dark, the next ghost that appears will be set to appear in a darker location.

[0220] Furthermore, this system also includes a means for creating an effect in which a ghostly hand appears from a specific object when the user approaches the object in the virtual real space. For example, it is possible to create a scenario in which a ghostly hand reaches out from a specific product shelf in a virtual store when the user approaches the product.

[0221] The virtual reality system also has a means for generating a darkness effect and making a ghost appear in a specific area in the virtual real space when the user performs a certain action in that area. For example, when the user enters a specific area in the virtual real space, the lighting in that area suddenly becomes dark and a ghost appears.

[0222] As a specific example, when a user approaches a specific product in a virtual store, a ghost's hand may reach out from under the product.Also, when a user approaches a specific area in the virtual store, the lights in that area may suddenly dim and a ghost may appear.

[0223] Examples of prompt sentences include:

[0224] "Create a scenario in a virtual store where ghostly hands reach out from the shelves when the user approaches a particular product."

[0225] "Create a horror scenario where when a user enters a certain area, the area becomes dark and ghosts appear."

[0226] In this way, it is possible to provide a highly interactive and immersive horror experience in both the user's real environment and the virtual real space.

[0227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0228] Step 1:

[0229] The server receives and analyzes environmental data at the user's residence.

[0230] Input: Photos and floor plan data of the residence uploaded by the user through a dedicated app

[0231] What it does: It receives this data and uses image analysis algorithms to determine the room layout and furniture placement.

[0232] Output: Room layout and furniture placement information

[0233] Step 2:

[0234] Based on the analysis results, the server generates ghosts and horror scenarios that are appropriate for the user's residence.

[0235] Input: Room layout and furniture placement information

[0236] Specific operation: Generate the location and performance details for ghost appearances for each room, for example, creating a scenario in which a ghost's hand reaches out from under the sofa.

[0237] Output: Ghost and horror scenario information

[0238] Step 3:

[0239] The server transmits the generated ghost and scenario information to the terminal.

[0240] Input: Ghost or horror scenario information

[0241] Specific operation: The server packages the information and sends it to the user's device through a dedicated application.

[0242] Output: Scenario information sent to the device

[0243] Step 4:

[0244] The terminal recognizes the user's home environment and the user's behavior.

[0245] Input: Received scenario information, real-time data from the device's camera and sensors

[0246] Specific operation: Using the device's camera and sensors, the device detects the user's location, movements, room brightness, etc. in real time and analyzes them.

[0247] Output: User's current location and movement information

[0248] Step 5:

[0249] The device will display the ghost at the appropriate time and place.

[0250] Input: User's current location and movement information, received scenario information

[0251] Specific operation: When the user approaches a location specified in the scenario, a ghostly hand or other effect will be displayed. For example, a ghostly hand will appear reaching out from under the sofa.

[0252] Output: The displayed ghost scene

[0253] Step 6:

[0254] The terminal monitors and collects the user's reaction data.

[0255] Input: User reaction data captured by camera and microphone (e.g., vocal responses, facial expressions)

[0256] What it does: The device uses its camera and microphone to record the user's reactions and sends the data to a server.

[0257] Output: Collected user response data

[0258] Step 7:

[0259] The server analyzes the collected reaction data and dynamically updates the horror scenario.

[0260] Input: Collected user response data

[0261] Specific behavior: Identify the timing and elements that made the user feel particularly scared, and adjust the scenario accordingly. For example, if the user is very scared of the dark, the next scenario will have ghosts appear from even darker locations.

[0262] Output: Updated horror scenario

[0263] Step 8:

[0264] The device will then display ghosts at the appropriate times and locations based on the new horror scenario, providing a continuous horror experience.

[0265] Input: Updated horror scenario

[0266] Specific behavior: The device receives the new scenario and displays ghosts again based on the user's actions, for example, adding new ghost effects to dark areas.

[0267] Output: The new ghost scene displayed

[0268] In this way, by linking each step together, it is possible to provide an interactive and immersive horror experience in both the user's home environment and the virtual real world.

[0269] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0270] The present invention is a system that utilizes environmental data and emotional data from a user's home to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[0271] Server Operation

[0272] Receiving and analyzing environmental data

[0273] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[0274] Generating ghosts and horror scenarios

[0275] The server generates ghost and horror scenarios that are optimal for the user's environment based on the analyzed residential information and emotion data obtained from the emotion engine. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[0276] Sending scenario information

[0277] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[0278] Device behavior

[0279] environmental awareness

[0280] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[0281] Displaying ghosts and running scenarios

[0282] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[0283] User behavior and sentiment monitoring

[0284] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user shouts or shows surprise, the device records their voice and movements, and the emotion engine analyzes the user's emotions from their facial expressions and voice. This data is then sent to the server.

[0285] Feedback and scenario updates

[0286] User response and emotion analysis

[0287] The server analyzes the user's reaction and emotional data sent from the device. This analysis identifies factors that frighten the user and changes in their emotions, such as fear of the dark or being startled by sudden movements.

[0288] Dynamic scenario updates

[0289] The server dynamically updates the horror scenario based on the analysis results. For example, it may set the next ghost to appear in a darker location, and adjusts in real time based on the emotion engine data. This provides the optimal horror experience according to the user's reactions and emotions.

[0290] Providing a continuous horror experience

[0291] The updated scenario information is then sent back to the device, which then receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[0292] Specific examples

[0293] For example, let's say a user starts an experience in a living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Using this information, the server generates a scenario in which a ghost's hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghost's hand at the appropriate time. The user jumps in surprise at this effect, and the device analyzes their facial expression using its emotion engine. The server adjusts the next scenario based on the analysis results and displays a scenario in which a ghost peeks out from behind the bookshelf.

[0294] In summary, the present invention is a system that provides a realistic and immersive horror experience based on the user's living environment and emotions.

[0295] The processing flow will be explained below.

[0296] Step 1:

[0297] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[0298] Step 2:

[0299] Device: Sends uploaded photos and floor plan data to the server.

[0300] Step 3:

[0301] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[0302] Step 4:

[0303] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[0304] Step 5:

[0305] Server: Sends the generated ghosts and scenario information to the user's device.

[0306] Step 6:

[0307] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[0308] Step 7:

[0309] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[0310] Step 8:

[0311] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or expresses surprise.

[0312] Step 9:

[0313] Device: Using the user's emotion engine, emotions such as fear and surprise are analyzed from the user's facial expressions and voice.

[0314] Step 10:

[0315] Terminal: Emotion data analyzed by the emotion engine is sent to the server.

[0316] Step 11:

[0317] Server: Analyzes the user's reaction and emotional data sent to identify elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[0318] Step 12:

[0319] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost to appear will be set to appear from a darker location.

[0320] Step 13:

[0321] Server: Sends updated scenario information to the device.

[0322] Step 14:

[0323] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[0324] As described above, the present invention is a system that dynamically generates a horror scenario based on the user's living environment and uses an emotion engine to analyze the user's emotions in real time, thereby providing an individually optimized interactive horror experience.

[0325] Example 2

[0326] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0327] Conventional horror experience systems have difficulty reflecting the user's living environment and individual emotional state in real time, making it impossible to provide a unique and immersive horror experience for the user. Furthermore, with static scenarios, the sense of fear fades after a scene is experienced once, making it difficult to maintain sustained excitement.

[0328] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data related to the user's living space, means for generating a virtual character and a horror scenario suited to the user's living space based on the analysis results, and means for transmitting the generated character and scenario information to the communication terminal. This enables a highly realistic and individually optimized horror experience tailored to the user's environment.

[0329] "User" refers to an individual who uses this system.

[0330] "Living space" refers to the home or room where a user lives their daily life.

[0331] "Environmental data" refers to information about the user's living space, such as photographs, floor plans, and furniture layout.

[0332] "Analysis" refers to a series of processes that analyze information based on received environmental data and identify the room layout and furniture placement.

[0333] "Virtual character" refers to a fear-inducing artificial character, such as a ghost, that appears in the user's living space.

[0334] A "horror scenario" refers to a sequence or performance designed to make the user feel fear.

[0335] "Communication terminal" refers to electronic devices used by users, such as smartphones and tablets.

[0336] "Monitoring" refers to the continuous observation of a user's behavior and reactions.

[0337] "Collect" refers to recording and storing user response data.

[0338] "Dynamic updates" refers to changing and adjusting scenarios and information in real time.

[0339] The present invention provides a system for providing an interactive horror experience using environmental data and emotional data in a user's living space. Specific embodiments of the system will be described in detail below.

[0340] Server Operation

[0341] The server first receives photos and floor plan data uploaded by the user through a dedicated app. This environmental data is sent to the server in JSON format. The server then uses a machine learning model (e.g., YOLOv5) to analyze the furniture placement and room layout from the received image data. The analysis results are stored in a database.

[0342] Next, emotional data is collected. The server uses an emotion analysis API (e.g., Azure Emotion API) to analyze the user's past reaction data. This allows the server to evaluate the situations in which the user feels particularly scared.

[0343] Based on the analyzed environmental data and emotional data, the server uses a generative AI model (e.g., GPT-4) to generate a horror scenario. The prompt includes instructions such as, "Please generate a scenario that will frighten the user." The generated scenario includes detailed information about the locations and timing of virtual character appearances. This scenario information is then encoded again into JSON format and sent to the user's communication device.

[0344] Device behavior

[0345] The device is equipped with a camera and sensors and scans the user's living environment in real time. The scan data is stored in the cloud and sent to the server as needed. The device also displays a virtual character in the user's real environment based on the horror scenario received from the server. Specifically, the device uses an augmented reality (AR) framework (e.g., ARKit) to overlay the virtual character on the user's camera image.

[0346] The device also uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to the server.

[0347] Feedback and scenario updates

[0348] The server analyzes the user's reaction data sent from the device and identifies elements that are particularly likely to cause fear. Based on the analysis results, a new prompt is given to the generative AI model to generate the next scenario. For example, the prompt might be, "Please generate a situation that makes the user feel even more scared."

[0349] The new scenario is then sent back to the device, which receives it and provides the user with a new horror experience. This feedback loop ensures that the user always receives a fresh, individually optimized horror experience.

[0350] Specific examples

[0351] For example, if a user wants to start the experience in their living room, they first launch a dedicated app and upload a photo of their living room to the server. The server analyzes the received data and identifies the locations of the bookshelf and sofa. The generative AI model then generates a scenario by providing a prompt: "Generate a scenario in which a ghost's hand reaches out from under the sofa." Based on this scenario, the device recognizes the user approaching the sofa and displays a virtual character. If the user jumps in surprise, their facial expression and voice are recorded and sent to the server. The server then analyzes this reaction data and generates the next scenario using a new prompt: "Generate a scenario in which a ghost peeks out from behind the bookshelf."

[0352] The above is a specific example of an embodiment of the present invention. This system allows users to enjoy a highly realistic and individually optimized horror experience.

[0353] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0354] Step 1:

[0355] The server receives photos and floor plan data of the living space uploaded by the user through a dedicated app. This input data includes images in JPEG and PNG format and floor plans in JSON format. The received data is stored in a database and prepared for analysis.

[0356] Step 2:

[0357] The server analyzes the received living space data. Specifically, it uses a machine learning model (e.g., YOLOv5) to identify furniture placement and room layout from the images. The analysis results are output in JSON format and stored in a database. This identifies the location and type of each piece of furniture in the room.

[0358] Step 3:

[0359] The server uses an emotion analysis API (e.g., Azure Emotion API) to collect user emotion data. The input data includes past user reaction data and audio and video data. The emotion analysis engine analyzes this data and evaluates the situations in which the user feels particularly scared. The evaluation results are output in JSON format and saved as analysis results.

[0360] Step 4:

[0361] The server uses a generative AI model (e.g., GPT-4) to generate a horror scenario based on the analyzed environmental and emotional data. The prompt is "Please generate a scenario that will frighten the user." The environmental and emotional data are provided as input to the generative AI model, and the generated horror scenario is obtained as output. This scenario is output in JSON format, detailing the locations and timing of character appearances.

[0362] Step 5:

[0363] The server sends the generated horror scenario to the user's communication device. The scenario information is encoded in JSON format and sent to the device via an HTTP POST request. The device analyzes the received data and makes preparations.

[0364] Step 6:

[0365] The device uses cameras and sensors to scan the user's living environment in real time, and uses LiDAR sensors and RGB cameras to perform 3D mapping to obtain detailed layout data of the room the user is in. The scanned data is then stored in the cloud in real time.

[0366] Step 7:

[0367] The device displays a virtual character in the user's real-world environment based on the horror scenario received from the server. Using an augmented reality (AR) framework (e.g., ARKit), the virtual character is overlaid on the user's camera image. Information about the character's appearance location and the content of the presentation is processed and displayed in real time.

[0368] Step 8:

[0369] The device uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to a server.

[0370] Step 9:

[0371] The server analyzes the user's reaction data sent from the device and identifies the elements that cause particular fear. It uses an emotion analysis API to analyze the reaction data and saves the analysis results in JSON format. This allows the user's fear triggers to be identified.

[0372] Step 10:

[0373] The server generates the next horror scenario by providing a new prompt to the generative AI model based on the analysis results. The prompt uses "Please generate a situation that will make the user feel even more scared." The server provides the analysis results and environmental data as input data, and obtains a new scenario as output.

[0374] Step 11:

[0375] The server then sends the generated new scenario to the communication terminal again. The scenario information is encoded in JSON format and sent to the terminal via an HTTP POST request, providing the user with a new horror experience.

[0376] The above are the specific processing steps of the program of this system.

[0377] (Application example 2)

[0378] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0379] In recent years, with the spread of virtual stores, there has been a demand for improved user experience. However, conventional virtual stores lack a system that can analyze users' emotions and reactions in real time and provide an optimal entertainment experience accordingly. The lack of such a system prevents users from getting an immersive experience, which reduces the appeal of virtual stores.

[0380] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and horror scenarios appropriate for the user's residence based on the analysis results, and means for transmitting the generated ghosts and scenario information to the terminal. This makes it possible to analyze the user's emotional data in real time and display ghosts at appropriate times and locations in the virtual store. In addition, by including means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for displaying ghosts at appropriate times and locations in the virtual environment as the user walks around the virtual store environment, and means for analyzing the user's emotional data in real time and using it to generate new horror scenarios, it is possible to provide an interactive horror experience that responds to the user's actions and emotions.

[0381] "Environmental data in the user's home" is data collected through cameras and sensors, including information on the layout, objects, brightness, temperature, and other aspects of the user's home.

[0382] "Means for analysis" refers to a processing device or software that uses algorithms or artificial intelligence to analyze collected environmental data and emotional data and extract specific information.

[0383] "Ghosts and horror scenarios" refer to virtual supernatural phenomena and the circumstances under which they appear that are designed to frighten users.

[0384] The "means for generating" is a processing device or software that creates new ghosts and horror scenarios based on the analysis results and emotional data.

[0385] The "transmitting means" is a communication device or software for transmitting the generated ghost and horror scenario information from the server to the user's terminal.

[0386] "Terminal" refers to electronic devices used by users, such as smartphones, tablets, and head-mounted displays (HMDs).

[0387] A "means for recognizing user behavior" is a device or software that uses a camera or sensor to detect the user's movements and location and analyzes that information.

[0388] The "means for displaying ghosts at the appropriate time and place" refers to a processing device or software that displays ghosts or horror scenarios at the appropriate time and place based on the user's behavior and location information in order to provide an optimal frightening experience.

[0389] "User reaction data" is data that records the user's reactions to the horror scenario, such as facial expressions, voice, and movements.

[0390] "Monitoring and collecting means" refers to devices or software for monitoring user responses in real time and collecting that data.

[0391] The "dynamic updating means" is a processing device or software for changing or evolving the current horror scenario based on collected user reaction data.

[0392] A "virtual store environment" is a virtual store interior space that is displayed on a device used by a user.

[0393] "Emotion data" is data that represents the user's emotional state analyzed from facial expressions, voice, body movements, and the like.

[0394] In this invention, the following hardware and software are used to build a system that utilizes the user's living environment and emotional data to provide an interactive horror experience in real time.

[0395] Server Operation

[0396] 1. Receiving and analyzing environmental data:

[0397] The server first receives photos and floor plan data of the home uploaded by the user through a dedicated app.

[0398] The received data is analyzed using image analysis software (e.g., OpenCV) to determine the room layout and furniture placement.

[0399] 2. Ghost and Horror Scenarios Generation:

[0400] The server generates ghost and horror scenarios that are optimal for the user's environment based on the detected residential information and emotion data obtained from the emotion engine.

[0401] The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the performance.

[0402] 3. Send scenario information:

[0403] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[0404] Device behavior

[0405] 1. Environmental awareness:

[0406] The device uses cameras and sensors (e.g., a smartphone's built-in camera and accelerometer) to scan the user's living environment in real time.

[0407] This allows the system to recognize the brightness of the room, the layout of furniture, and the user's position.

[0408] 2. Displaying ghosts and running scenarios:

[0409] The terminal displays the ghost superimposed on the user's real environment based on the ghost and scenario information sent from the server.

[0410] For example, when a user is sitting on a sofa, a ghost's hand is displayed in real time from under the sofa.

[0411] 3. User behavior and sentiment monitoring:

[0412] The device uses a camera and microphone to monitor the user's actions and reactions in real time.

[0413] When a user screams or gets surprised, their voice and movements are recorded, and the emotion engine analyzes the user's emotions from their facial expressions and voice.

[0414] Feedback and scenario updates

[0415] 1. User response and sentiment analysis:

[0416] The server analyzes the user's reaction data and emotion data sent from the terminal.

[0417] This analysis identifies factors that the user finds particularly frightening and emotional fluctuations.

[0418] 2. Dynamic scenario update:

[0419] The server dynamically updates the horror scenario based on the analysis results.

[0420] For example, the next ghost will be set to appear from a darker location, and adjustments will be made in real time based on data from the emotion engine.

[0421] Specific use cases

[0422] As users walk around the virtual store and look at the products, they are presented with a horror experience in which a ghost peeks out from behind a shelf. When the user shows signs of surprise and their facial expression changes, the emotion analysis engine recognizes their reaction. In the next scenario, the user is provided with even more surprising elements.

[0423] Prompt Sentence Examples

[0424] "Users were surprised by the scenario where a ghost was peeking out from behind a shelf, so in the next scenario, add a scene where the lights go out and something is hiding under the shelf."

[0425] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0426] Step 1:

[0427] The server receives photos and floor plan data of the home uploaded by the user through a dedicated app. The input data are image files and drawing files provided by the user. To analyze this data, the server uses image analysis software (e.g., OpenCV) to identify the room layout and furniture placement. The output is room layout information and furniture position information.

[0428] Step 2:

[0429] The server generates ghosts and horror scenarios based on the analysis results and emotional data obtained from the user's emotion engine. The input data is the layout information of the residence and emotional data. Using a generative AI model, this data is analyzed to generate optimal ghosts and scenarios. The output is details of ghosts and horror scenarios that are suitable for the user's residence environment.

[0430] Step 3:

[0431] The server sends the generated ghost and scenario information to the user's device. The input data is detailed information about the ghost and scenario, and the output is data sent to the device. The server synchronizes the information using a communication protocol (e.g., HTTP).

[0432] Step 4:

[0433] The device uses cameras and sensors to scan the user's living environment in real time. The input data is on-site video and environmental data acquired by the cameras and sensors. The output data is real-time information about the room's brightness, furniture layout, and the user's location.

[0434] Step 5:

[0435] The device displays ghosts superimposed on the user's real environment based on ghost and scenario information sent from the server. The input data is detailed information about ghosts and scenarios sent from the server, as well as real-time environmental data. The output is ghosts and horror effects superimposed on the user's real environment.

[0436] Step 6:

[0437] The device uses a camera and microphone to monitor and collect the user's actions and reactions in real time. The input data is the user's audio and video information. The output is reaction data such as the user's facial expressions, voice, and movements.

[0438] Step 7:

[0439] The device analyzes the user's emotional data using an emotion engine. The input data is the user's reaction data, and the output is the analyzed user's emotional state. The emotion engine uses facial expression recognition and voice analysis algorithms.

[0440] Step 8:

[0441] The server dynamically updates the horror scenario based on the user's reaction data and emotional data sent from the device. The input data is the user's reaction data and emotional state data. The output is detailed information about the new horror scenario. A new optimal scenario is generated using a generative AI model.

[0442] Step 9:

[0443] The server then sends the updated new scenario information to the device again. The input data is the dynamically adjusted horror scenario details, and the output is the data sent to the device. This information is displayed appropriately on the device, providing the user with a continuous horror experience.

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0445] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0447] [Second embodiment]

[0448] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0449] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0450] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0452] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0454] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0455] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0456] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0459] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0460] The present invention provides an interactive horror experience by utilizing environmental data of the user's home. The program and processing of this system will be described in detail below.

[0461] Server Operation

[0462] Receiving and analyzing environmental data

[0463] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[0464] Generating ghosts and horror scenarios

[0465] The server generates ghost and horror scenarios that best fit the user's environment based on the analyzed residential information. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[0466] Sending scenario information

[0467] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[0468] Device behavior

[0469] environmental awareness

[0470] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[0471] Displaying ghosts and running scenarios

[0472] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[0473] User behavior monitoring

[0474] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the device records the voice and movements and sends them to the server. This data is important for showing when the user felt particularly scared.

[0475] Feedback and scenario updates

[0476] Analysis of user responses and scenario updates

[0477] The server analyzes the user's reaction data sent from the device and identifies the elements that frighten the user. For example, if the user is particularly afraid of the dark, the next ghost displayed will be set to appear from a darker location. In this way, the scenario is dynamically updated, aiming to maximize the sense of fear.

[0478] Providing a continuous horror experience

[0479] The updated scenario information is then sent back to the device, which then receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[0480] Specific examples

[0481] For example, let's say a user starts an experience in their living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghostly hand at the appropriate time. If the user jumps in surprise at this effect, the video and audio of the user's reaction are recorded and sent to the server. The server adjusts the next scenario based on this reaction, continuously evolving the terrifying experience.

[0482] As described above, the present invention is a system that dynamically links with the user's home environment to provide a realistic and immersive horror experience.

[0483] The processing flow will be explained below.

[0484] Step 1:

[0485] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[0486] Step 2:

[0487] Device: Sends uploaded photos and floor plan data to the server.

[0488] Step 3:

[0489] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[0490] Step 4:

[0491] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[0492] Step 5:

[0493] Server: Sends the generated ghosts and scenario information to the user's device.

[0494] Step 6:

[0495] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[0496] Step 7:

[0497] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[0498] Step 8:

[0499] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or expresses surprise.

[0500] Step 9:

[0501] Terminal: Sends user reaction data to the server.

[0502] Step 10:

[0503] Server: Analyzes the user's reaction data and identifies the elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[0504] Step 11:

[0505] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost may appear from a darker location.

[0506] Step 12:

[0507] Server: Sends updated scenario information to the device.

[0508] Step 13:

[0509] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[0510] In this way, the present invention provides a dynamic and individually optimized horror experience based on the user's living environment.

[0511] Example 1

[0512] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0513] Conventional horror experience systems have had the problem of being unable to provide an immersive experience, as it is difficult to provide a frightening experience that corresponds to the user's individual living environment and behavior. Furthermore, it is difficult to analyze the user's reactions in real time and dynamically update the scenario. This creates the problem of users becoming bored.

[0514] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0515] In this invention, the server includes means for receiving and analyzing environmental information about the user's residence, means for generating a virtual image and a horror experience scenario suited to the user's residence based on the analysis results, and means for transmitting the generated virtual image and scenario information to the information terminal. This makes it possible to provide a horror experience suited to the user's individual environment. The information terminal also includes means for detecting the user's residence environment and user behavior and displaying a virtual image at an appropriate time and place, means for monitoring and collecting user reaction data, and means for analyzing the collected reaction data and dynamically updating the horror experience scenario. This makes it possible to individually optimize the scenario based on the user's reaction and provide a sustained and immersive horror experience.

[0516] 1. "User's place of residence" refers to the place where the user lives on a daily basis, including the environment such as a house or apartment.

[0517] 2. "Environmental information" refers to information about the user's place of residence, including data such as room layout, furniture arrangement, brightness, temperature, and sound.

[0518] 3. "Virtual images" refer to images or visual effects that do not exist in reality but are visually displayed, including horror effects such as ghosts.

[0519] 4. "Horror experience scenario" refers to a series of actions or storylines designed to instill fear in the user.

[0520] 5. "Information terminal" refers to a device used by a user, including a smartphone, tablet, computer, etc.

[0521] 6. “Analysis” means the process or method of examining collected data in detail and extracting necessary information.

[0522] 7. "Detection" refers to sensing the environment or user behavior using devices such as sensors and cameras.

[0523] 8. "Monitoring" means continuously observing user behavior and reactions and obtaining necessary data.

[0524] 9. "Collection" refers to the compilation and retention of data obtained through surveillance.

[0525] 10. "Dynamic updates" refers to changing the scenario or presentation in real time according to the situation.

[0526] 11. "Individual optimization" means providing optimal content tailored to the characteristics of each user based on collected data.

[0527] 12. “Generative AI Model” means a computational model that uses artificial intelligence techniques to generate new data or content.

[0528] The present invention is a system that provides an interactive horror experience by utilizing environmental information of a user's residence. This system realizes an immersive horror experience in real time using a terminal, a server, and cameras and sensors for monitoring the user's actions and reactions. Specific embodiments of this system are described below.

[0529] Server Operation

[0530] Receiving and analyzing environmental information

[0531] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app. After receiving the data, the server analyzes this data to determine the room layout and furniture placement information. For example, if a sofa or bookshelf is located in the living room, it will identify their locations. Image recognition algorithms are used for the analysis.

[0532] Generating ghosts and horror scenarios

[0533] Based on the analyzed residential information, the server generates virtual images and horror scenarios that are optimal for the user's environment. The generative AI model is used to devise scenarios and determine the timing and content of the scenes. For example, it can generate scenes in which a ghostly hand reaches out from under a sofa or a closet door opens on its own.

[0534] Sending scenario information

[0535] The generated virtual images and horror scenarios are sent from the server to the information terminal, where data is communicated and synchronized in real time to ensure a smooth user experience.

[0536] Device behavior

[0537] environmental awareness

[0538] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the room's brightness, furniture layout, and the user's location, and sends the information to the server. For example, the device may detect that the user is near a bookshelf.

[0539] Displaying ghosts and running scenarios

[0540] Based on the scenario information sent from the server, the device displays a virtual image superimposed on the user's real environment. For example, if the user is sitting on a sofa, a ghost's hand will appear to reach out from under the sofa.

[0541] User behavior monitoring

[0542] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the voice and movement are recorded and sent to the server. This allows the device to understand when the user is feeling particularly scared.

[0543] Feedback and scenario updates

[0544] Analysis of user responses and scenario updates

[0545] The server analyzes the user's reaction data sent from the device and identifies the elements of fear that should be enhanced in the next scenario. For example, if the user has a strong fear of the dark, the server will set the next scenario to have ghosts appear from darker places.

[0546] Providing a continuous horror experience

[0547] The server then sends the updated scenario information back to the device, which then receives it and provides a new horror experience, ensuring that users always receive a fresh, individually optimized horror experience.

[0548] Specific examples

[0549] 1. The user begins the experience in their living room

[0550] The server analyzes photos of the living room and identifies the location of the bookshelf and sofa.

[0551] Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device.

[0552] 2. Device-based environment recognition and scenario execution

[0553] The device uses cameras and sensors to scan the user's living environment and detects when the user approaches the sofa.

[0554] A ghostly hand appears at the appropriate time to create a frightening effect.

[0555] 3. User response and data transmission

[0556] When the user screams in surprise, their voice and movements are recorded by the terminal.

[0557] The terminal transmits this reaction data to the server.

[0558] 4. Feedback analysis and scenario update

[0559] The server analyzes the user's reaction data and identifies areas that need to be strengthened in the next scenario. For example, if darkness causes fear, the next performance could have a virtual image appear from a dark place.

[0560] 5. Submit new scenarios and continue the experience

[0561] The server then sends the updated scenario back to the device, which receives it and provides a new horror experience.

[0562] Prompt Sentence Examples

[0563] "Generate a scenario where something crawls out from under the sofa the moment the user enters the living room. Think of a realistic effect that will surprise the user."

[0564] "Create a scenario where a ghost appears from the shadow of the bookshelf when the user goes to get a book. Set the timing to make the user feel scared."

[0565] "I thought of an eerie sound playing in the dark, and when the user heads in the direction of the sound, a ghost will appear."

[0566] As described above, the present invention dynamically links with the user's home environment to provide an individually optimized horror experience.

[0567] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0568] Program processing flow

[0569] Server Operation

[0570] Step 1:

[0571] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app.

[0572] Input: Photo and floor plan data of the residence submitted by the user.

[0573] Output: Received residence location data.

[0574] Specific operation: Use the file receiving module to obtain image data from the user.

[0575] Step 2:

[0576] The server inputs the received data into an image recognition algorithm for analysis.

[0577] Input: Received residential location photo and floor plan data.

[0578] Output: Analyzed room layout and furniture placement information.

[0579] What it does: Uses image recognition software to identify furniture locations and room layouts.

[0580] Step 3:

[0581] The server uses a generative AI model to generate virtual images and horror experience scenarios based on the analysis results.

[0582] Input: Room layout and furniture placement information.

[0583] Output: Generated virtual images and frightening scenarios.

[0584] Specific operation: A prompt sentence is input into the generative AI model to generate a scenario. For example, it generates a scenario in which a ghost's hand reaches out from under the sofa.

[0585] Step 4:

[0586] The server transmits the generated virtual image and scenario information to the user's terminal.

[0587] Input: Generated virtual image and scenario information.

[0588] Output: Scenario data sent to the user's information terminal.

[0589] Specific operation: The scenario data is transmitted to the user terminal using the data transmission module.

[0590] Device behavior

[0591] Step 5:

[0592] The device uses cameras and sensors to scan the user's living environment in real time and transmits the data to a server.

[0593] Input: Real-world environment data acquired by cameras and sensors.

[0594] Output: The environment data sent to the server.

[0595] Specific operation: The environmental recognition module is used to detect the room brightness, furniture arrangement, and user position in real time.

[0596] Step 6:

[0597] Based on the scenario information sent from the server, the terminal displays a virtual image superimposed on the user's real environment.

[0598] Input: Scenario information sent from the server.

[0599] Output: Displayed virtual image and frightening experience scenario.

[0600] Specific operation: Using the AR (Augmented Reality) module, a ghost hand appears from under the sofa when the user approaches it.

[0601] Step 7:

[0602] The device uses a camera and microphone to monitor the user's actions and reactions in real time and transmits the data to a server.

[0603] Input: User behavior and voice data captured by camera and microphone.

[0604] Output: User response data sent to the server.

[0605] Specific actions: Use the monitoring module to record the user's voice and actions such as shouting or being surprised.

[0606] Feedback and scenario updates

[0607] Step 8:

[0608] The server analyzes the user's reaction data sent from the device and identifies the fear elements that should be strengthened in the next scenario.

[0609] Input: User reaction data sent from the device.

[0610] Output: Scenario with new and enhanced horror elements based on analysis.

[0611] Specific behavior: Use the data analysis module to identify the moments and elements that made users feel most scared.

[0612] Step 9:

[0613] The server generates new scenarios and sends them to the user's device, continually updating the terrifying experience.

[0614] Input: User's reaction data analysis results.

[0615] Output: New scenario data.

[0616] Specific operation: New conditions are input into the generative AI model to generate scenarios with enhanced fear elements, such as "a scenario in which a ghost appears from a dark place."

[0617] As described above, this system allows the server and device to cooperate to provide users with an individually optimized interactive horror experience.

[0618] (Application example 1)

[0619] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0620] Conventional horror experience systems do not adequately generate horror scenarios that are dependent on the user's real-world environment, nor do they dynamically update the scenarios based on the user's real-time actions and reactions. As a result, there are issues with the user's sense of fear and immersion being limited. In addition, it has been difficult to provide a horror experience through interaction in a virtual real space.

[0621] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0622] In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and a horror scenario appropriate for the user's residence based on the analysis results, means for transmitting the generated ghost and scenario information to the terminal, means for recognizing the user's residence environment and user behavior in the terminal and displaying a ghost at an appropriate time and place, means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for causing a ghost's hand to appear from a specific object when the user approaches the object in the virtual real space, and means for generating a darkness effect and causing a ghost to appear in that location when the user performs a certain action in a specific area in the virtual real space, thereby providing an interactive and immersive horror experience in both the user's real environment and the virtual real space.

[0623] "Environmental data in the user's residence" is data including the layout, furniture arrangement, lighting conditions, and other related information within the user's residence.

[0624] "Ghost and Horror Scenarios" are scenes of mysterious phenomena and terrifying experiences that are generated based on the user's interactions in their living environment and virtual real space.

[0625] "Terminals" are devices such as smartphones, smart glasses, and head-mounted displays that allow users to experience horror scenarios visually and aurally.

[0626] "Means for recognizing user behavior" refers to a function that detects information such as the user's position, movements, voice, and facial expressions in real time.

[0627] "User reaction data" refers to data including behavior, vocal responses, and physiological responses such as heart rate that a user exhibits in response to a horror scenario.

[0628] "Means for dynamically updating horror scenarios" is a function that analyzes collected user reaction data and changes the content and presentation of the scenario to provide the user with the optimal horror experience.

[0629] A "virtual real space" is a virtual reality environment, a digital space designed for users to experience.

[0630] The "effect of a ghostly hand appearing from the object" is an effect in which, when a user approaches a specific virtual object, a mysterious phenomenon (for example, a ghostly hand) visually occurs from the object.

[0631] "Means for creating a darkness effect and making a ghost appear in that location" refers to a production in which, when a user performs a specific action in a specific area of ​​the virtual real space, the lights in that area are dimmed and a ghost appears.

[0632] This invention is a system that utilizes data on a user's home environment to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[0633] The server has a means for receiving and analyzing environmental data of the user's residence, including photos and floor plan data of the user's residence. Based on this data, the server analyzes the room layout and furniture arrangement to generate specific ghost and horror scenarios.

[0634] Based on the analysis results, the server has the means to generate ghosts and horror scenarios that are appropriate for the user's residence. For example, if the user has a sofa in their living room, the server will generate a scenario in which a ghost's hand reaches out from under the sofa. This generated ghost and scenario information is sent to the device via a dedicated application.

[0635] The device has a means of recognizing the user's home environment and behavior. The device is equipped with a camera and sensors, and can detect the user's location and behavior in real time. For example, when the user approaches a specific location, that information is sent to the server.

[0636] The device also has a means to display ghosts at the appropriate time and place. Based on the scenario information sent from the server, the device can display ghosts superimposed on the user's real environment. For example, when the user is sitting on a sofa, the device will display an effect in which a ghost's hand reaches out from under the sofa.

[0637] Additionally, the device has a means of monitoring and collecting user reaction data. It uses a camera and microphone to record the user's actions and reactions and transmits the data to a server. This reaction data includes when the user shouts or shows surprise.

[0638] The server analyzes the collected reaction data and dynamically updates the horror scenario. It identifies the elements that frighten the user most and reflects this in the next scenario. For example, if the user is very scared of the dark, the next ghost that appears will be set to appear in a darker location.

[0639] Furthermore, this system also includes a means for creating an effect in which a ghostly hand appears from a specific object when the user approaches the object in the virtual real space. For example, it is possible to create a scenario in which a ghostly hand reaches out from a specific product shelf in a virtual store when the user approaches the product.

[0640] The virtual reality system also has a means for generating a darkness effect and making a ghost appear in a specific area in the virtual real space when the user performs a certain action in that area. For example, when the user enters a specific area in the virtual real space, the lighting in that area suddenly becomes dark and a ghost appears.

[0641] As a specific example, when a user approaches a specific product in a virtual store, a ghost's hand may reach out from under the product.Also, when a user approaches a specific area in the virtual store, the lights in that area may suddenly dim and a ghost may appear.

[0642] Examples of prompt sentences include:

[0643] "Create a scenario in a virtual store where ghostly hands reach out from the shelves when the user approaches a particular product."

[0644] "Create a horror scenario where when a user enters a certain area, the area becomes dark and ghosts appear."

[0645] In this way, it is possible to provide a highly interactive and immersive horror experience in both the user's real environment and the virtual real space.

[0646] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0647] Step 1:

[0648] The server receives and analyzes environmental data at the user's residence.

[0649] Input: Photos and floor plan data of the residence uploaded by the user through a dedicated app

[0650] What it does: It receives this data and uses image analysis algorithms to determine the room layout and furniture placement.

[0651] Output: Room layout and furniture placement information

[0652] Step 2:

[0653] Based on the analysis results, the server generates ghosts and horror scenarios that are appropriate for the user's residence.

[0654] Input: Room layout and furniture placement information

[0655] Specific operation: Generate the location and performance details for ghost appearances for each room, for example, creating a scenario in which a ghost's hand reaches out from under the sofa.

[0656] Output: Ghost and horror scenario information

[0657] Step 3:

[0658] The server transmits the generated ghost and scenario information to the terminal.

[0659] Input: Ghost or horror scenario information

[0660] Specific operation: The server packages the information and sends it to the user's device through a dedicated application.

[0661] Output: Scenario information sent to the device

[0662] Step 4:

[0663] The terminal recognizes the user's home environment and the user's behavior.

[0664] Input: Received scenario information, real-time data from the device's camera and sensors

[0665] Specific operation: Using the device's camera and sensors, the device detects the user's location, movements, room brightness, etc. in real time and analyzes them.

[0666] Output: User's current location and movement information

[0667] Step 5:

[0668] The device will display the ghost at the appropriate time and place.

[0669] Input: User's current location and movement information, received scenario information

[0670] Specific operation: When the user approaches a location specified in the scenario, a ghostly hand or other effect will be displayed. For example, a ghostly hand will appear reaching out from under the sofa.

[0671] Output: The displayed ghost scene

[0672] Step 6:

[0673] The terminal monitors and collects the user's reaction data.

[0674] Input: User reaction data captured by camera and microphone (e.g., vocal responses, facial expressions)

[0675] What it does: The device uses its camera and microphone to record the user's reactions and sends the data to a server.

[0676] Output: Collected user response data

[0677] Step 7:

[0678] The server analyzes the collected reaction data and dynamically updates the horror scenario.

[0679] Input: Collected user response data

[0680] Specific behavior: Identify the timing and elements that made the user feel particularly scared, and adjust the scenario accordingly. For example, if the user is very scared of the dark, the next scenario will have ghosts appear from even darker locations.

[0681] Output: Updated horror scenario

[0682] Step 8:

[0683] The device will then display ghosts at the appropriate times and locations based on the new horror scenario, providing a continuous horror experience.

[0684] Input: Updated horror scenario

[0685] Specific behavior: The device receives the new scenario and displays ghosts again based on the user's actions, for example, adding new ghost effects to dark areas.

[0686] Output: The new ghost scene displayed

[0687] In this way, by linking each step together, it is possible to provide an interactive and immersive horror experience in both the user's home environment and the virtual real world.

[0688] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0689] The present invention is a system that utilizes environmental data and emotional data from a user's home to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[0690] Server Operation

[0691] Receiving and analyzing environmental data

[0692] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[0693] Generating ghosts and horror scenarios

[0694] The server generates ghost and horror scenarios that are optimal for the user's environment based on the analyzed residential information and emotion data obtained from the emotion engine. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[0695] Sending scenario information

[0696] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[0697] Device behavior

[0698] environmental awareness

[0699] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[0700] Displaying ghosts and running scenarios

[0701] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[0702] User behavior and sentiment monitoring

[0703] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user shouts or shows surprise, the device records their voice and movements, and the emotion engine analyzes the user's emotions from their facial expressions and voice. This data is then sent to the server.

[0704] Feedback and scenario updates

[0705] User response and emotion analysis

[0706] The server analyzes the user's reaction and emotional data sent from the device. This analysis identifies factors that frighten the user and changes in their emotions, such as fear of the dark or being startled by sudden movements.

[0707] Dynamic scenario updates

[0708] The server dynamically updates the horror scenario based on the analysis results. For example, it may set the next ghost to appear in a darker location, and adjusts in real time based on the emotion engine data. This provides the optimal horror experience according to the user's reactions and emotions.

[0709] Providing a continuous horror experience

[0710] The updated scenario information is then sent back to the device, which then receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[0711] Specific examples

[0712] For example, let's say a user starts an experience in a living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Using this information, the server generates a scenario in which a ghost's hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghost's hand at the appropriate time. The user jumps in surprise at this effect, and the device analyzes their facial expression using its emotion engine. The server adjusts the next scenario based on the analysis results and displays a scenario in which a ghost peeks out from behind the bookshelf.

[0713] In summary, the present invention is a system that provides a realistic and immersive horror experience based on the user's living environment and emotions.

[0714] The processing flow will be explained below.

[0715] Step 1:

[0716] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[0717] Step 2:

[0718] Device: Sends uploaded photos and floor plan data to the server.

[0719] Step 3:

[0720] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[0721] Step 4:

[0722] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[0723] Step 5:

[0724] Server: Sends the generated ghosts and scenario information to the user's device.

[0725] Step 6:

[0726] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[0727] Step 7:

[0728] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[0729] Step 8:

[0730] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or expresses surprise.

[0731] Step 9:

[0732] Device: Using the user's emotion engine, emotions such as fear and surprise are analyzed from the user's facial expressions and voice.

[0733] Step 10:

[0734] Terminal: Emotion data analyzed by the emotion engine is sent to the server.

[0735] Step 11:

[0736] Server: Analyzes the user's reaction and emotional data sent to identify elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[0737] Step 12:

[0738] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost to appear will be set to appear from a darker location.

[0739] Step 13:

[0740] Server: Sends updated scenario information to the device.

[0741] Step 14:

[0742] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[0743] As described above, the present invention is a system that dynamically generates a horror scenario based on the user's living environment and uses an emotion engine to analyze the user's emotions in real time, thereby providing an individually optimized interactive horror experience.

[0744] Example 2

[0745] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] Conventional horror experience systems have difficulty reflecting the user's living environment and individual emotional state in real time, making it impossible to provide a unique and immersive horror experience for the user. Furthermore, with static scenarios, the sense of fear fades after a scene is experienced once, making it difficult to maintain sustained excitement.

[0747] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data related to the user's living space, means for generating a virtual character and a horror scenario suited to the user's living space based on the analysis results, and means for transmitting the generated character and scenario information to the communication terminal. This enables a highly realistic and individually optimized horror experience tailored to the user's environment.

[0748] "User" refers to an individual who uses this system.

[0749] "Living space" refers to the home or room where a user lives their daily life.

[0750] "Environmental data" refers to information about the user's living space, such as photographs, floor plans, and furniture layout.

[0751] "Analysis" refers to a series of processes that analyze information based on received environmental data and identify the room layout and furniture placement.

[0752] "Virtual character" refers to a fear-inducing artificial character, such as a ghost, that appears in the user's living space.

[0753] A "horror scenario" refers to a sequence or performance designed to make the user feel fear.

[0754] "Communication terminal" refers to electronic devices used by users, such as smartphones and tablets.

[0755] "Monitoring" refers to the continuous observation of a user's behavior and reactions.

[0756] "Collect" refers to recording and storing user response data.

[0757] "Dynamic updates" refers to changing and adjusting scenarios and information in real time.

[0758] The present invention provides a system for providing an interactive horror experience using environmental data and emotional data in a user's living space. Specific embodiments of the system will be described in detail below.

[0759] Server Operation

[0760] The server first receives photos and floor plan data uploaded by the user through a dedicated app. This environmental data is sent to the server in JSON format. The server then uses a machine learning model (e.g., YOLOv5) to analyze the furniture placement and room layout from the received image data. The analysis results are stored in a database.

[0761] Next, emotional data is collected. The server uses an emotion analysis API (e.g., Azure Emotion API) to analyze the user's past reaction data. This allows the server to evaluate the situations in which the user feels particularly scared.

[0762] Based on the analyzed environmental data and emotional data, the server uses a generative AI model (e.g., GPT-4) to generate a horror scenario. The prompt includes instructions such as, "Please generate a scenario that will frighten the user." The generated scenario includes detailed information about the locations and timing of virtual character appearances. This scenario information is then encoded again into JSON format and sent to the user's communication device.

[0763] Device behavior

[0764] The device is equipped with a camera and sensors and scans the user's living environment in real time. The scan data is stored in the cloud and sent to the server as needed. The device also displays a virtual character in the user's real environment based on the horror scenario received from the server. Specifically, the device uses an augmented reality (AR) framework (e.g., ARKit) to overlay the virtual character on the user's camera image.

[0765] The device also uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to the server.

[0766] Feedback and scenario updates

[0767] The server analyzes the user's reaction data sent from the device and identifies elements that are particularly likely to cause fear. Based on the analysis results, a new prompt is given to the generative AI model to generate the next scenario. For example, the prompt might be, "Please generate a situation that makes the user feel even more scared."

[0768] The new scenario is then sent back to the device, which receives it and provides the user with a new horror experience. This feedback loop ensures that the user always receives a fresh, individually optimized horror experience.

[0769] Specific examples

[0770] For example, if a user wants to start the experience in their living room, they first launch a dedicated app and upload a photo of their living room to the server. The server analyzes the received data and identifies the locations of the bookshelf and sofa. The generative AI model then generates a scenario by providing a prompt: "Generate a scenario in which a ghost's hand reaches out from under the sofa." Based on this scenario, the device recognizes the user approaching the sofa and displays a virtual character. If the user jumps in surprise, their facial expression and voice are recorded and sent to the server. The server then analyzes this reaction data and generates the next scenario using a new prompt: "Generate a scenario in which a ghost peeks out from behind the bookshelf."

[0771] The above is a specific example of an embodiment of the present invention. This system allows users to enjoy a highly realistic and individually optimized horror experience.

[0772] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0773] Step 1:

[0774] The server receives photos and floor plan data of the living space uploaded by the user through a dedicated app. This input data includes images in JPEG and PNG format and floor plans in JSON format. The received data is stored in a database and prepared for analysis.

[0775] Step 2:

[0776] The server analyzes the received living space data. Specifically, it uses a machine learning model (e.g., YOLOv5) to identify furniture placement and room layout from the images. The analysis results are output in JSON format and stored in a database. This identifies the location and type of each piece of furniture in the room.

[0777] Step 3:

[0778] The server uses an emotion analysis API (e.g., Azure Emotion API) to collect user emotion data. The input data includes past user reaction data and audio and video data. The emotion analysis engine analyzes this data and evaluates the situations in which the user feels particularly scared. The evaluation results are output in JSON format and saved as analysis results.

[0779] Step 4:

[0780] The server uses a generative AI model (e.g., GPT-4) to generate a horror scenario based on the analyzed environmental and emotional data. The prompt is "Please generate a scenario that will frighten the user." The environmental and emotional data are provided as input to the generative AI model, and the generated horror scenario is obtained as output. This scenario is output in JSON format, detailing the locations and timing of character appearances.

[0781] Step 5:

[0782] The server sends the generated horror scenario to the user's communication device. The scenario information is encoded in JSON format and sent to the device via an HTTP POST request. The device analyzes the received data and makes preparations.

[0783] Step 6:

[0784] The device uses cameras and sensors to scan the user's living environment in real time, and uses LiDAR sensors and RGB cameras to perform 3D mapping to obtain detailed layout data of the room the user is in. The scanned data is then stored in the cloud in real time.

[0785] Step 7:

[0786] The device displays a virtual character in the user's real-world environment based on the horror scenario received from the server. Using an augmented reality (AR) framework (e.g., ARKit), the virtual character is overlaid on the user's camera image. Information about the character's appearance location and the content of the presentation is processed and displayed in real time.

[0787] Step 8:

[0788] The device uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to a server.

[0789] Step 9:

[0790] The server analyzes the user's reaction data sent from the device and identifies the elements that cause particular fear. It uses an emotion analysis API to analyze the reaction data and saves the analysis results in JSON format. This allows the user's fear triggers to be identified.

[0791] Step 10:

[0792] The server generates the next horror scenario by providing a new prompt to the generative AI model based on the analysis results. The prompt uses "Please generate a situation that will make the user feel even more scared." The server provides the analysis results and environmental data as input data, and obtains a new scenario as output.

[0793] Step 11:

[0794] The server then sends the generated new scenario to the communication terminal again. The scenario information is encoded in JSON format and sent to the terminal via an HTTP POST request, providing the user with a new horror experience.

[0795] The above are the specific processing steps of the program of this system.

[0796] (Application example 2)

[0797] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0798] In recent years, with the spread of virtual stores, there has been a demand for improved user experience. However, conventional virtual stores lack a system that can analyze users' emotions and reactions in real time and provide an optimal entertainment experience accordingly. The lack of such a system prevents users from getting an immersive experience, which reduces the appeal of virtual stores.

[0799] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and horror scenarios appropriate for the user's residence based on the analysis results, and means for transmitting the generated ghosts and scenario information to the terminal. This makes it possible to analyze the user's emotional data in real time and display ghosts at appropriate times and locations in the virtual store. In addition, by including means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for displaying ghosts at appropriate times and locations in the virtual environment as the user walks around the virtual store environment, and means for analyzing the user's emotional data in real time and using it to generate new horror scenarios, it is possible to provide an interactive horror experience that responds to the user's actions and emotions.

[0800] "Environmental data in the user's home" is data collected through cameras and sensors, including information on the layout, objects, brightness, temperature, and other aspects of the user's home.

[0801] "Means for analysis" refers to a processing device or software that uses algorithms or artificial intelligence to analyze collected environmental data and emotional data and extract specific information.

[0802] "Ghosts and horror scenarios" refer to virtual supernatural phenomena and the circumstances under which they appear that are designed to frighten users.

[0803] The "means for generating" is a processing device or software that creates new ghosts and horror scenarios based on the analysis results and emotional data.

[0804] The "transmitting means" is a communication device or software for transmitting the generated ghost and horror scenario information from the server to the user's terminal.

[0805] "Terminal" refers to electronic devices used by users, such as smartphones, tablets, and head-mounted displays (HMDs).

[0806] A "means for recognizing user behavior" is a device or software that uses a camera or sensor to detect the user's movements and location and analyzes that information.

[0807] The "means for displaying ghosts at the appropriate time and place" refers to a processing device or software that displays ghosts or horror scenarios at the appropriate time and place based on the user's behavior and location information in order to provide an optimal frightening experience.

[0808] "User reaction data" is data that records the user's reactions to the horror scenario, such as facial expressions, voice, and movements.

[0809] "Monitoring and collecting means" refers to devices or software for monitoring user responses in real time and collecting that data.

[0810] The "dynamic updating means" is a processing device or software for changing or evolving the current horror scenario based on collected user reaction data.

[0811] A "virtual store environment" is a virtual store interior space that is displayed on a device used by a user.

[0812] "Emotion data" is data that represents the user's emotional state analyzed from facial expressions, voice, body movements, and the like.

[0813] In this invention, the following hardware and software are used to build a system that utilizes the user's living environment and emotional data to provide an interactive horror experience in real time.

[0814] Server Operation

[0815] 1. Receiving and analyzing environmental data:

[0816] The server first receives photos and floor plan data of the home uploaded by the user through a dedicated app.

[0817] The received data is analyzed using image analysis software (e.g., OpenCV) to determine the room layout and furniture placement.

[0818] 2. Ghost and Horror Scenarios Generation:

[0819] The server generates ghost and horror scenarios that are optimal for the user's environment based on the detected residential information and emotion data obtained from the emotion engine.

[0820] The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the performance.

[0821] 3. Send scenario information:

[0822] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[0823] Device behavior

[0824] 1. Environmental awareness:

[0825] The device uses cameras and sensors (e.g., a smartphone's built-in camera and accelerometer) to scan the user's living environment in real time.

[0826] This allows the system to recognize the brightness of the room, the layout of furniture, and the user's position.

[0827] 2. Displaying ghosts and running scenarios:

[0828] The terminal displays the ghost superimposed on the user's real environment based on the ghost and scenario information sent from the server.

[0829] For example, when a user is sitting on a sofa, a ghost's hand is displayed in real time from under the sofa.

[0830] 3. User behavior and sentiment monitoring:

[0831] The device uses a camera and microphone to monitor the user's actions and reactions in real time.

[0832] When a user screams or gets surprised, their voice and movements are recorded, and the emotion engine analyzes the user's emotions from their facial expressions and voice.

[0833] Feedback and scenario updates

[0834] 1. User response and sentiment analysis:

[0835] The server analyzes the user's reaction data and emotion data sent from the terminal.

[0836] This analysis identifies factors that the user finds particularly frightening and emotional fluctuations.

[0837] 2. Dynamic scenario update:

[0838] The server dynamically updates the horror scenario based on the analysis results.

[0839] For example, the next ghost will be set to appear from a darker location, and adjustments will be made in real time based on data from the emotion engine.

[0840] Specific use cases

[0841] As users walk around the virtual store and look at the products, they are presented with a horror experience in which a ghost peeks out from behind a shelf. When the user shows signs of surprise and their facial expression changes, the emotion analysis engine recognizes their reaction. In the next scenario, the user is provided with even more surprising elements.

[0842] Prompt Sentence Examples

[0843] "Users were surprised by the scenario where a ghost was peeking out from behind a shelf, so in the next scenario, add a scene where the lights go out and something is hiding under the shelf."

[0844] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0845] Step 1:

[0846] The server receives photos and floor plan data of the home uploaded by the user through a dedicated app. The input data are image files and drawing files provided by the user. To analyze this data, the server uses image analysis software (e.g., OpenCV) to identify the room layout and furniture placement. The output is room layout information and furniture position information.

[0847] Step 2:

[0848] The server generates ghosts and horror scenarios based on the analysis results and emotional data obtained from the user's emotion engine. The input data is the layout information of the residence and emotional data. Using a generative AI model, this data is analyzed to generate optimal ghosts and scenarios. The output is details of ghosts and horror scenarios that are suitable for the user's residence environment.

[0849] Step 3:

[0850] The server sends the generated ghost and scenario information to the user's device. The input data is detailed information about the ghost and scenario, and the output is data sent to the device. The server synchronizes the information using a communication protocol (e.g., HTTP).

[0851] Step 4:

[0852] The device uses cameras and sensors to scan the user's living environment in real time. The input data is on-site video and environmental data acquired by the cameras and sensors. The output data is real-time information about the room's brightness, furniture layout, and the user's location.

[0853] Step 5:

[0854] The device displays ghosts superimposed on the user's real environment based on ghost and scenario information sent from the server. The input data is detailed information about ghosts and scenarios sent from the server, as well as real-time environmental data. The output is ghosts and horror effects superimposed on the user's real environment.

[0855] Step 6:

[0856] The device uses a camera and microphone to monitor and collect the user's actions and reactions in real time. The input data is the user's audio and video information. The output is reaction data such as the user's facial expressions, voice, and movements.

[0857] Step 7:

[0858] The device analyzes the user's emotional data using an emotion engine. The input data is the user's reaction data, and the output is the analyzed user's emotional state. The emotion engine uses facial expression recognition and voice analysis algorithms.

[0859] Step 8:

[0860] The server dynamically updates the horror scenario based on the user's reaction data and emotional data sent from the device. The input data is the user's reaction data and emotional state data. The output is detailed information about the new horror scenario. A new optimal scenario is generated using a generative AI model.

[0861] Step 9:

[0862] The server then sends the updated new scenario information to the device again. The input data is the dynamically adjusted horror scenario details, and the output is the data sent to the device. This information is displayed appropriately on the device, providing the user with a continuous horror experience.

[0863] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0864] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0865] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0866] [Third embodiment]

[0867] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0868] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0869] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0870] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0871] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0872] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0873] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0874] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0875] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0876] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0877] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0878] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0879] The present invention provides an interactive horror experience by utilizing environmental data of the user's home. The program and processing of this system will be described in detail below.

[0880] Server Operation

[0881] Receiving and analyzing environmental data

[0882] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[0883] Generating ghosts and horror scenarios

[0884] The server generates ghost and horror scenarios that best fit the user's environment based on the analyzed residential information. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[0885] Sending scenario information

[0886] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[0887] Device behavior

[0888] environmental awareness

[0889] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[0890] Displaying ghosts and running scenarios

[0891] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[0892] User behavior monitoring

[0893] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the device records the voice and movements and sends them to the server. This data is important for showing when the user felt particularly scared.

[0894] Feedback and scenario updates

[0895] Analysis of user responses and scenario updates

[0896] The server analyzes the user's reaction data sent from the device and identifies the elements that frighten the user. For example, if the user is particularly afraid of the dark, the next ghost displayed will be set to appear from a darker location. In this way, the scenario is dynamically updated, aiming to maximize the sense of fear.

[0897] Providing a continuous horror experience

[0898] The updated scenario information is then sent back to the device, which then receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[0899] Specific examples

[0900] For example, let's say a user starts an experience in their living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghostly hand at the appropriate time. If the user jumps in surprise at this effect, the video and audio of the user's reaction are recorded and sent to the server. The server adjusts the next scenario based on this reaction, continuously evolving the terrifying experience.

[0901] As described above, the present invention is a system that dynamically links with the user's home environment to provide a realistic and immersive horror experience.

[0902] The processing flow will be explained below.

[0903] Step 1:

[0904] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[0905] Step 2:

[0906] Device: Sends uploaded photos and floor plan data to the server.

[0907] Step 3:

[0908] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[0909] Step 4:

[0910] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[0911] Step 5:

[0912] Server: Sends the generated ghosts and scenario information to the user's device.

[0913] Step 6:

[0914] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[0915] Step 7:

[0916] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[0917] Step 8:

[0918] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or expresses surprise.

[0919] Step 9:

[0920] Terminal: Sends user reaction data to the server.

[0921] Step 10:

[0922] Server: Analyzes the user's reaction data and identifies the elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[0923] Step 11:

[0924] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost may appear from a darker location.

[0925] Step 12:

[0926] Server: Sends updated scenario information to the device.

[0927] Step 13:

[0928] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[0929] In this way, the present invention provides a dynamic and individually optimized horror experience based on the user's living environment.

[0930] Example 1

[0931] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0932] Conventional horror experience systems have had the problem of being unable to provide an immersive experience, as it is difficult to provide a frightening experience that corresponds to the user's individual living environment and behavior. Furthermore, it is difficult to analyze the user's reactions in real time and dynamically update the scenario. This creates the problem of users becoming bored.

[0933] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0934] In this invention, the server includes means for receiving and analyzing environmental information about the user's residence, means for generating a virtual image and a horror experience scenario suited to the user's residence based on the analysis results, and means for transmitting the generated virtual image and scenario information to the information terminal. This makes it possible to provide a horror experience suited to the user's individual environment. The information terminal also includes means for detecting the user's residence environment and user behavior and displaying a virtual image at an appropriate time and place, means for monitoring and collecting user reaction data, and means for analyzing the collected reaction data and dynamically updating the horror experience scenario. This makes it possible to individually optimize the scenario based on the user's reaction and provide a sustained and immersive horror experience.

[0935] 1. "User's place of residence" refers to the place where the user lives on a daily basis, including the environment such as a house or apartment.

[0936] 2. "Environmental information" refers to information about the user's place of residence, including data such as room layout, furniture arrangement, brightness, temperature, and sound.

[0937] 3. "Virtual images" refer to images or visual effects that do not exist in reality but are visually displayed, including horror effects such as ghosts.

[0938] 4. "Horror experience scenario" refers to a series of actions or storylines designed to instill fear in the user.

[0939] 5. "Information terminal" refers to a device used by a user, including a smartphone, tablet, computer, etc.

[0940] 6. “Analysis” means the process or method of examining collected data in detail and extracting necessary information.

[0941] 7. "Detection" refers to sensing the environment or user behavior using devices such as sensors and cameras.

[0942] 8. "Monitoring" means continuously observing user behavior and reactions and obtaining necessary data.

[0943] 9. "Collection" refers to the compilation and retention of data obtained through surveillance.

[0944] 10. "Dynamic updates" refers to changing the scenario or presentation in real time according to the situation.

[0945] 11. "Individual optimization" means providing optimal content tailored to the characteristics of each user based on collected data.

[0946] 12. “Generative AI Model” means a computational model that uses artificial intelligence techniques to generate new data or content.

[0947] The present invention is a system that provides an interactive horror experience by utilizing environmental information of a user's residence. This system realizes an immersive horror experience in real time using a terminal, a server, and cameras and sensors for monitoring the user's actions and reactions. Specific embodiments of this system are described below.

[0948] Server Operation

[0949] Receiving and analyzing environmental information

[0950] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app. After receiving the data, the server analyzes this data to determine the room layout and furniture placement information. For example, if a sofa or bookshelf is located in the living room, it will identify their locations. Image recognition algorithms are used for the analysis.

[0951] Generating ghosts and horror scenarios

[0952] Based on the analyzed residential information, the server generates virtual images and horror scenarios that are optimal for the user's environment. The generative AI model is used to devise scenarios and determine the timing and content of the scenes. For example, it can generate scenes in which a ghostly hand reaches out from under a sofa or a closet door opens on its own.

[0953] Sending scenario information

[0954] The generated virtual images and horror scenarios are sent from the server to the information terminal, where data is communicated and synchronized in real time to ensure a smooth user experience.

[0955] Device behavior

[0956] environmental awareness

[0957] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the room's brightness, furniture layout, and the user's location, and sends the information to the server. For example, the device may detect that the user is near a bookshelf.

[0958] Displaying ghosts and running scenarios

[0959] Based on the scenario information sent from the server, the device displays a virtual image superimposed on the user's real environment. For example, if the user is sitting on a sofa, a ghost's hand will appear to reach out from under the sofa.

[0960] User behavior monitoring

[0961] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the voice and movement are recorded and sent to the server. This allows the device to understand when the user is feeling particularly scared.

[0962] Feedback and scenario updates

[0963] Analysis of user responses and scenario updates

[0964] The server analyzes the user's reaction data sent from the device and identifies the elements of fear that should be enhanced in the next scenario. For example, if the user has a strong fear of the dark, the server will set the next scenario to have ghosts appear from darker places.

[0965] Providing a continuous horror experience

[0966] The server then sends the updated scenario information back to the device, which then receives it and provides a new horror experience, ensuring that users always receive a fresh, individually optimized horror experience.

[0967] Specific examples

[0968] 1. The user begins the experience in their living room

[0969] The server analyzes photos of the living room and identifies the location of the bookshelf and sofa.

[0970] Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device.

[0971] 2. Device-based environment recognition and scenario execution

[0972] The device uses cameras and sensors to scan the user's living environment and detects when the user approaches the sofa.

[0973] A ghostly hand appears at the appropriate time to create a frightening effect.

[0974] 3. User response and data transmission

[0975] When the user screams in surprise, their voice and movements are recorded by the terminal.

[0976] The terminal transmits this reaction data to the server.

[0977] 4. Feedback analysis and scenario update

[0978] The server analyzes the user's reaction data and identifies areas that need to be strengthened in the next scenario. For example, if darkness causes fear, the next performance could have a virtual image appear from a dark place.

[0979] 5. Submit new scenarios and continue the experience

[0980] The server then sends the updated scenario back to the device, which receives it and provides a new horror experience.

[0981] Prompt Sentence Examples

[0982] "Generate a scenario where something crawls out from under the sofa the moment the user enters the living room. Think of a realistic effect that will surprise the user."

[0983] "Create a scenario where a ghost appears from the shadow of the bookshelf when the user goes to get a book. Set the timing to make the user feel scared."

[0984] "I thought of an eerie sound playing in the dark, and when the user heads in the direction of the sound, a ghost will appear."

[0985] As described above, the present invention dynamically links with the user's home environment to provide an individually optimized horror experience.

[0986] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0987] Program processing flow

[0988] Server Operation

[0989] Step 1:

[0990] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app.

[0991] Input: Photo and floor plan data of the residence submitted by the user.

[0992] Output: Received residence location data.

[0993] Specific operation: Use the file receiving module to obtain image data from the user.

[0994] Step 2:

[0995] The server inputs the received data into an image recognition algorithm for analysis.

[0996] Input: Received residential location photo and floor plan data.

[0997] Output: Analyzed room layout and furniture placement information.

[0998] What it does: Uses image recognition software to identify furniture locations and room layouts.

[0999] Step 3:

[1000] The server uses a generative AI model to generate virtual images and horror experience scenarios based on the analysis results.

[1001] Input: Room layout and furniture placement information.

[1002] Output: Generated virtual images and frightening scenarios.

[1003] Specific operation: A prompt sentence is input into the generative AI model to generate a scenario. For example, it generates a scenario in which a ghost's hand reaches out from under the sofa.

[1004] Step 4:

[1005] The server transmits the generated virtual image and scenario information to the user's terminal.

[1006] Input: Generated virtual image and scenario information.

[1007] Output: Scenario data sent to the user's information terminal.

[1008] Specific operation: The scenario data is transmitted to the user terminal using the data transmission module.

[1009] Device behavior

[1010] Step 5:

[1011] The device uses cameras and sensors to scan the user's living environment in real time and transmits the data to a server.

[1012] Input: Real-world environment data acquired by cameras and sensors.

[1013] Output: The environment data sent to the server.

[1014] Specific operation: The environmental recognition module is used to detect the room brightness, furniture arrangement, and user position in real time.

[1015] Step 6:

[1016] Based on the scenario information sent from the server, the terminal displays a virtual image superimposed on the user's real environment.

[1017] Input: Scenario information sent from the server.

[1018] Output: Displayed virtual image and frightening experience scenario.

[1019] Specific operation: Using the AR (Augmented Reality) module, a ghost hand appears from under the sofa when the user approaches it.

[1020] Step 7:

[1021] The device uses a camera and microphone to monitor the user's actions and reactions in real time and transmits the data to a server.

[1022] Input: User behavior and voice data captured by camera and microphone.

[1023] Output: User response data sent to the server.

[1024] Specific actions: Use the monitoring module to record the user's voice and actions such as shouting or being surprised.

[1025] Feedback and scenario updates

[1026] Step 8:

[1027] The server analyzes the user's reaction data sent from the device and identifies the fear elements that should be strengthened in the next scenario.

[1028] Input: User reaction data sent from the device.

[1029] Output: Scenario with new and enhanced horror elements based on analysis.

[1030] Specific behavior: Use the data analysis module to identify the moments and elements that made users feel most scared.

[1031] Step 9:

[1032] The server generates new scenarios and sends them to the user's device, continually updating the terrifying experience.

[1033] Input: User's reaction data analysis results.

[1034] Output: New scenario data.

[1035] Specific operation: New conditions are input into the generative AI model to generate scenarios with enhanced fear elements, such as "a scenario in which a ghost appears from a dark place."

[1036] As described above, this system allows the server and device to cooperate to provide users with an individually optimized interactive horror experience.

[1037] (Application example 1)

[1038] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1039] Conventional horror experience systems do not adequately generate horror scenarios that are dependent on the user's real-world environment, nor do they dynamically update the scenarios based on the user's real-time actions and reactions. As a result, there are issues with the user's sense of fear and immersion being limited. In addition, it has been difficult to provide a horror experience through interaction in a virtual real space.

[1040] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1041] In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and a horror scenario appropriate for the user's residence based on the analysis results, means for transmitting the generated ghost and scenario information to the terminal, means for recognizing the user's residence environment and user behavior in the terminal and displaying a ghost at an appropriate time and place, means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for causing a ghost's hand to appear from a specific object when the user approaches the object in the virtual real space, and means for generating a darkness effect and causing a ghost to appear in that location when the user performs a certain action in a specific area in the virtual real space, thereby providing an interactive and immersive horror experience in both the user's real environment and the virtual real space.

[1042] "Environmental data in the user's residence" is data including the layout, furniture arrangement, lighting conditions, and other related information within the user's residence.

[1043] "Ghost and Horror Scenarios" are scenes of mysterious phenomena and terrifying experiences that are generated based on the user's interactions in their living environment and virtual real space.

[1044] "Terminals" are devices such as smartphones, smart glasses, and head-mounted displays that allow users to experience horror scenarios visually and aurally.

[1045] "Means for recognizing user behavior" refers to a function that detects information such as the user's position, movements, voice, and facial expressions in real time.

[1046] "User reaction data" refers to data including behavior, vocal responses, and physiological responses such as heart rate that a user exhibits in response to a horror scenario.

[1047] "Means for dynamically updating horror scenarios" is a function that analyzes collected user reaction data and changes the content and presentation of the scenario to provide the user with the optimal horror experience.

[1048] A "virtual real space" is a virtual reality environment, a digital space designed for users to experience.

[1049] The "effect of a ghostly hand appearing from the object" is an effect in which, when a user approaches a specific virtual object, a mysterious phenomenon (for example, a ghostly hand) visually occurs from the object.

[1050] "Means for creating a darkness effect and making a ghost appear in that location" refers to a production in which, when a user performs a specific action in a specific area of ​​the virtual real space, the lights in that area are dimmed and a ghost appears.

[1051] This invention is a system that utilizes data on a user's home environment to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[1052] The server has a means for receiving and analyzing environmental data of the user's residence, including photos and floor plan data of the user's residence. Based on this data, the server analyzes the room layout and furniture arrangement to generate specific ghost and horror scenarios.

[1053] Based on the analysis results, the server has the means to generate ghosts and horror scenarios that are appropriate for the user's residence. For example, if the user has a sofa in their living room, the server will generate a scenario in which a ghost's hand reaches out from under the sofa. This generated ghost and scenario information is sent to the device via a dedicated application.

[1054] The device has a means of recognizing the user's home environment and behavior. The device is equipped with a camera and sensors, and can detect the user's location and behavior in real time. For example, when the user approaches a specific location, that information is sent to the server.

[1055] The device also has a means to display ghosts at the appropriate time and place. Based on the scenario information sent from the server, the device can display ghosts superimposed on the user's real environment. For example, when the user is sitting on a sofa, the device will display an effect in which a ghost's hand reaches out from under the sofa.

[1056] Additionally, the device has a means of monitoring and collecting user reaction data. It uses a camera and microphone to record the user's actions and reactions and transmits the data to a server. This reaction data includes when the user shouts or shows surprise.

[1057] The server analyzes the collected reaction data and dynamically updates the horror scenario. It identifies the elements that frighten the user most and reflects this in the next scenario. For example, if the user is very scared of the dark, the next ghost that appears will be set to appear in a darker location.

[1058] Furthermore, this system also includes a means for creating an effect in which a ghostly hand appears from a specific object when the user approaches the object in the virtual real space. For example, it is possible to create a scenario in which a ghostly hand reaches out from a specific product shelf in a virtual store when the user approaches the product.

[1059] The virtual reality system also has a means for generating a darkness effect and making a ghost appear in a specific area in the virtual real space when the user performs a certain action in that area. For example, when the user enters a specific area in the virtual real space, the lighting in that area suddenly becomes dark and a ghost appears.

[1060] As a specific example, when a user approaches a specific product in a virtual store, a ghost's hand may reach out from under the product.Also, when a user approaches a specific area in the virtual store, the lights in that area may suddenly dim and a ghost may appear.

[1061] Examples of prompt sentences include:

[1062] "Create a scenario in a virtual store where ghostly hands reach out from the shelves when the user approaches a particular product."

[1063] "Create a horror scenario where when a user enters a certain area, the area becomes dark and ghosts appear."

[1064] In this way, it is possible to provide a highly interactive and immersive horror experience in both the user's real environment and the virtual real space.

[1065] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1066] Step 1:

[1067] The server receives and analyzes environmental data at the user's residence.

[1068] Input: Photos and floor plan data of the residence uploaded by the user through a dedicated app

[1069] What it does: It receives this data and uses image analysis algorithms to determine the room layout and furniture placement.

[1070] Output: Room layout and furniture placement information

[1071] Step 2:

[1072] Based on the analysis results, the server generates ghosts and horror scenarios that are appropriate for the user's residence.

[1073] Input: Room layout and furniture placement information

[1074] Specific operation: Generate the location and performance details for ghost appearances for each room, for example, creating a scenario in which a ghost's hand reaches out from under the sofa.

[1075] Output: Ghost and horror scenario information

[1076] Step 3:

[1077] The server transmits the generated ghost and scenario information to the terminal.

[1078] Input: Ghost or horror scenario information

[1079] Specific operation: The server packages the information and sends it to the user's device through a dedicated application.

[1080] Output: Scenario information sent to the device

[1081] Step 4:

[1082] The terminal recognizes the user's home environment and the user's behavior.

[1083] Input: Received scenario information, real-time data from the device's camera and sensors

[1084] Specific operation: Using the device's camera and sensors, the device detects the user's location, movements, room brightness, etc. in real time and analyzes them.

[1085] Output: User's current location and movement information

[1086] Step 5:

[1087] The device will display the ghost at the appropriate time and place.

[1088] Input: User's current location and movement information, received scenario information

[1089] Specific operation: When the user approaches a location specified in the scenario, a ghostly hand or other effect will be displayed. For example, a ghostly hand will appear reaching out from under the sofa.

[1090] Output: The displayed ghost scene

[1091] Step 6:

[1092] The terminal monitors and collects the user's reaction data.

[1093] Input: User reaction data captured by camera and microphone (e.g., vocal responses, facial expressions)

[1094] What it does: The device uses its camera and microphone to record the user's reactions and sends the data to a server.

[1095] Output: Collected user response data

[1096] Step 7:

[1097] The server analyzes the collected reaction data and dynamically updates the horror scenario.

[1098] Input: Collected user response data

[1099] Specific behavior: Identify the timing and elements that made the user feel particularly scared, and adjust the scenario accordingly. For example, if the user is very scared of the dark, the next scenario will have ghosts appear from even darker locations.

[1100] Output: Updated horror scenario

[1101] Step 8:

[1102] The device will then display ghosts at the appropriate times and locations based on the new horror scenario, providing a continuous horror experience.

[1103] Input: Updated horror scenario

[1104] Specific behavior: The device receives the new scenario and displays ghosts again based on the user's actions, for example, adding new ghost effects to dark areas.

[1105] Output: The new ghost scene displayed

[1106] In this way, by linking each step together, it is possible to provide an interactive and immersive horror experience in both the user's home environment and the virtual real world.

[1107] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1108] The present invention is a system that utilizes environmental data and emotional data from a user's home to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[1109] Server Operation

[1110] Receiving and analyzing environmental data

[1111] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[1112] Generating ghosts and horror scenarios

[1113] The server generates ghost and horror scenarios that are optimal for the user's environment based on the analyzed residential information and emotion data obtained from the emotion engine. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[1114] Sending scenario information

[1115] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[1116] Device behavior

[1117] environmental awareness

[1118] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[1119] Displaying ghosts and running scenarios

[1120] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[1121] User behavior and sentiment monitoring

[1122] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user shouts or shows surprise, the device records their voice and movements, and the emotion engine analyzes the user's emotions from their facial expressions and voice. This data is then sent to the server.

[1123] Feedback and scenario updates

[1124] User response and emotion analysis

[1125] The server analyzes the user's reaction and emotional data sent from the device. This analysis identifies factors that frighten the user and changes in their emotions, such as fear of the dark or being startled by sudden movements.

[1126] Dynamic scenario updates

[1127] The server dynamically updates the horror scenario based on the analysis results. For example, it may set the next ghost to appear in a darker location, and adjusts in real time based on the emotion engine data. This provides the optimal horror experience according to the user's reactions and emotions.

[1128] Providing a continuous horror experience

[1129] The updated scenario information is then sent back to the device, which then receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[1130] Specific examples

[1131] For example, let's say a user starts an experience in a living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Using this information, the server generates a scenario in which a ghost's hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghost's hand at the appropriate time. The user jumps in surprise at this effect, and the device analyzes their facial expression using its emotion engine. The server adjusts the next scenario based on the analysis results and displays a scenario in which a ghost peeks out from behind the bookshelf.

[1132] In summary, the present invention is a system that provides a realistic and immersive horror experience based on the user's living environment and emotions.

[1133] The processing flow will be explained below.

[1134] Step 1:

[1135] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[1136] Step 2:

[1137] Device: Sends uploaded photos and floor plan data to the server.

[1138] Step 3:

[1139] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[1140] Step 4:

[1141] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[1142] Step 5:

[1143] Server: Sends the generated ghosts and scenario information to the user's device.

[1144] Step 6:

[1145] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[1146] Step 7:

[1147] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[1148] Step 8:

[1149] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or expresses surprise.

[1150] Step 9:

[1151] Device: Using the user's emotion engine, emotions such as fear and surprise are analyzed from the user's facial expressions and voice.

[1152] Step 10:

[1153] Terminal: Emotion data analyzed by the emotion engine is sent to the server.

[1154] Step 11:

[1155] Server: Analyzes the user's reaction and emotional data sent to identify elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[1156] Step 12:

[1157] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost to appear will be set to appear from a darker location.

[1158] Step 13:

[1159] Server: Sends updated scenario information to the device.

[1160] Step 14:

[1161] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[1162] As described above, the present invention is a system that dynamically generates a horror scenario based on the user's living environment and uses an emotion engine to analyze the user's emotions in real time, thereby providing an individually optimized interactive horror experience.

[1163] Example 2

[1164] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1165] Conventional horror experience systems have difficulty reflecting the user's living environment and individual emotional state in real time, making it impossible to provide a unique and immersive horror experience for the user. Furthermore, with static scenarios, the sense of fear fades after a scene is experienced once, making it difficult to maintain sustained excitement.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data related to the user's living space, means for generating a virtual character and a horror scenario suited to the user's living space based on the analysis results, and means for transmitting the generated character and scenario information to the communication terminal. This enables a highly realistic and individually optimized horror experience tailored to the user's environment.

[1167] "User" refers to an individual who uses this system.

[1168] "Living space" refers to the home or room where a user lives their daily life.

[1169] "Environmental data" refers to information about the user's living space, such as photographs, floor plans, and furniture layout.

[1170] "Analysis" refers to a series of processes that analyze information based on received environmental data and identify the room layout and furniture placement.

[1171] "Virtual character" refers to a fear-inducing artificial character, such as a ghost, that appears in the user's living space.

[1172] A "horror scenario" refers to a sequence or performance designed to make the user feel fear.

[1173] "Communication terminal" refers to electronic devices used by users, such as smartphones and tablets.

[1174] "Monitoring" refers to the continuous observation of a user's behavior and reactions.

[1175] "Collect" refers to recording and storing user response data.

[1176] "Dynamic updates" refers to changing and adjusting scenarios and information in real time.

[1177] The present invention provides a system for providing an interactive horror experience using environmental data and emotional data in a user's living space. Specific embodiments of the system will be described in detail below.

[1178] Server Operation

[1179] The server first receives photos and floor plan data uploaded by the user through a dedicated app. This environmental data is sent to the server in JSON format. The server then uses a machine learning model (e.g., YOLOv5) to analyze the furniture placement and room layout from the received image data. The analysis results are stored in a database.

[1180] Next, emotional data is collected. The server uses an emotion analysis API (e.g., Azure Emotion API) to analyze the user's past reaction data. This allows the server to evaluate the situations in which the user feels particularly scared.

[1181] Based on the analyzed environmental data and emotional data, the server uses a generative AI model (e.g., GPT-4) to generate a horror scenario. The prompt includes instructions such as, "Please generate a scenario that will frighten the user." The generated scenario includes detailed information about the locations and timing of virtual character appearances. This scenario information is then encoded again into JSON format and sent to the user's communication device.

[1182] Device behavior

[1183] The device is equipped with a camera and sensors and scans the user's living environment in real time. The scan data is stored in the cloud and sent to the server as needed. The device also displays a virtual character in the user's real environment based on the horror scenario received from the server. Specifically, the device uses an augmented reality (AR) framework (e.g., ARKit) to overlay the virtual character on the user's camera image.

[1184] The device also uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to the server.

[1185] Feedback and scenario updates

[1186] The server analyzes the user's reaction data sent from the device and identifies elements that are particularly likely to cause fear. Based on the analysis results, a new prompt is given to the generative AI model to generate the next scenario. For example, the prompt might be, "Please generate a situation that makes the user feel even more scared."

[1187] The new scenario is then sent back to the device, which receives it and provides the user with a new horror experience. This feedback loop ensures that the user always receives a fresh, individually optimized horror experience.

[1188] Specific examples

[1189] For example, if a user wants to start the experience in their living room, they first launch a dedicated app and upload a photo of their living room to the server. The server analyzes the received data and identifies the locations of the bookshelf and sofa. The generative AI model then generates a scenario by providing a prompt: "Generate a scenario in which a ghost's hand reaches out from under the sofa." Based on this scenario, the device recognizes the user approaching the sofa and displays a virtual character. If the user jumps in surprise, their facial expression and voice are recorded and sent to the server. The server then analyzes this reaction data and generates the next scenario using a new prompt: "Generate a scenario in which a ghost peeks out from behind the bookshelf."

[1190] The above is a specific example of an embodiment of the present invention. This system allows users to enjoy a highly realistic and individually optimized horror experience.

[1191] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1192] Step 1:

[1193] The server receives photos and floor plan data of the living space uploaded by the user through a dedicated app. This input data includes images in JPEG and PNG format and floor plans in JSON format. The received data is stored in a database and prepared for analysis.

[1194] Step 2:

[1195] The server analyzes the received living space data. Specifically, it uses a machine learning model (e.g., YOLOv5) to identify furniture placement and room layout from the images. The analysis results are output in JSON format and stored in a database. This identifies the location and type of each piece of furniture in the room.

[1196] Step 3:

[1197] The server uses an emotion analysis API (e.g., Azure Emotion API) to collect user emotion data. The input data includes past user reaction data and audio and video data. The emotion analysis engine analyzes this data and evaluates the situations in which the user feels particularly scared. The evaluation results are output in JSON format and saved as analysis results.

[1198] Step 4:

[1199] The server uses a generative AI model (e.g., GPT-4) to generate a horror scenario based on the analyzed environmental and emotional data. The prompt is "Please generate a scenario that will frighten the user." The environmental and emotional data are provided as input to the generative AI model, and the generated horror scenario is obtained as output. This scenario is output in JSON format, detailing the locations and timing of character appearances.

[1200] Step 5:

[1201] The server sends the generated horror scenario to the user's communication device. The scenario information is encoded in JSON format and sent to the device via an HTTP POST request. The device analyzes the received data and makes preparations.

[1202] Step 6:

[1203] The device uses cameras and sensors to scan the user's living environment in real time, and uses LiDAR sensors and RGB cameras to perform 3D mapping to obtain detailed layout data of the room the user is in. The scanned data is then stored in the cloud in real time.

[1204] Step 7:

[1205] The device displays a virtual character in the user's real-world environment based on the horror scenario received from the server. Using an augmented reality (AR) framework (e.g., ARKit), the virtual character is overlaid on the user's camera image. Information about the character's appearance location and the content of the presentation is processed and displayed in real time.

[1206] Step 8:

[1207] The device uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to a server.

[1208] Step 9:

[1209] The server analyzes the user's reaction data sent from the device and identifies the elements that cause particular fear. It uses an emotion analysis API to analyze the reaction data and saves the analysis results in JSON format. This allows the user's fear triggers to be identified.

[1210] Step 10:

[1211] The server generates the next horror scenario by providing a new prompt to the generative AI model based on the analysis results. The prompt uses "Please generate a situation that will make the user feel even more scared." The server provides the analysis results and environmental data as input data, and obtains a new scenario as output.

[1212] Step 11:

[1213] The server then sends the generated new scenario to the communication terminal again. The scenario information is encoded in JSON format and sent to the terminal via an HTTP POST request, providing the user with a new horror experience.

[1214] The above are the specific processing steps of the program of this system.

[1215] (Application example 2)

[1216] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1217] In recent years, with the spread of virtual stores, there has been a demand for improved user experience. However, conventional virtual stores lack a system that can analyze users' emotions and reactions in real time and provide an optimal entertainment experience accordingly. The lack of such a system prevents users from getting an immersive experience, which reduces the appeal of virtual stores.

[1218] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and horror scenarios appropriate for the user's residence based on the analysis results, and means for transmitting the generated ghosts and scenario information to the terminal. This makes it possible to analyze the user's emotional data in real time and display ghosts at appropriate times and locations in the virtual store. In addition, by including means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for displaying ghosts at appropriate times and locations in the virtual environment as the user walks around the virtual store environment, and means for analyzing the user's emotional data in real time and using it to generate new horror scenarios, it is possible to provide an interactive horror experience that responds to the user's actions and emotions.

[1219] "Environmental data in the user's home" is data collected through cameras and sensors, including information on the layout, objects, brightness, temperature, and other aspects of the user's home.

[1220] "Means for analysis" refers to a processing device or software that uses algorithms or artificial intelligence to analyze collected environmental data and emotional data and extract specific information.

[1221] "Ghosts and horror scenarios" refer to virtual supernatural phenomena and the circumstances under which they appear that are designed to frighten users.

[1222] The "means for generating" is a processing device or software that creates new ghosts and horror scenarios based on the analysis results and emotional data.

[1223] The "transmitting means" is a communication device or software for transmitting the generated ghost and horror scenario information from the server to the user's terminal.

[1224] "Terminal" refers to electronic devices used by users, such as smartphones, tablets, and head-mounted displays (HMDs).

[1225] A "means for recognizing user behavior" is a device or software that uses a camera or sensor to detect the user's movements and location and analyzes that information.

[1226] The "means for displaying ghosts at the appropriate time and place" refers to a processing device or software that displays ghosts or horror scenarios at the appropriate time and place based on the user's behavior and location information in order to provide an optimal frightening experience.

[1227] "User reaction data" is data that records the user's reactions to the horror scenario, such as facial expressions, voice, and movements.

[1228] "Monitoring and collecting means" refers to devices or software for monitoring user responses in real time and collecting that data.

[1229] The "dynamic updating means" is a processing device or software for changing or evolving the current horror scenario based on collected user reaction data.

[1230] A "virtual store environment" is a virtual store interior space that is displayed on a device used by a user.

[1231] "Emotion data" is data that represents the user's emotional state analyzed from facial expressions, voice, body movements, and the like.

[1232] In this invention, the following hardware and software are used to build a system that utilizes the user's living environment and emotional data to provide an interactive horror experience in real time.

[1233] Server Operation

[1234] 1. Receiving and analyzing environmental data:

[1235] The server first receives photos and floor plan data of the home uploaded by the user through a dedicated app.

[1236] The received data is analyzed using image analysis software (e.g., OpenCV) to determine the room layout and furniture placement.

[1237] 2. Ghost and Horror Scenarios Generation:

[1238] The server generates ghost and horror scenarios that are optimal for the user's environment based on the detected residential information and emotion data obtained from the emotion engine.

[1239] The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the performance.

[1240] 3. Send scenario information:

[1241] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[1242] Device behavior

[1243] 1. Environmental awareness:

[1244] The device uses cameras and sensors (e.g., a smartphone's built-in camera and accelerometer) to scan the user's living environment in real time.

[1245] This allows the system to recognize the brightness of the room, the layout of furniture, and the user's position.

[1246] 2. Displaying ghosts and running scenarios:

[1247] The terminal displays the ghost superimposed on the user's real environment based on the ghost and scenario information sent from the server.

[1248] For example, when a user is sitting on a sofa, a ghost's hand is displayed in real time from under the sofa.

[1249] 3. User behavior and sentiment monitoring:

[1250] The device uses a camera and microphone to monitor the user's actions and reactions in real time.

[1251] When a user screams or gets surprised, their voice and movements are recorded, and the emotion engine analyzes the user's emotions from their facial expressions and voice.

[1252] Feedback and scenario updates

[1253] 1. User response and sentiment analysis:

[1254] The server analyzes the user's reaction data and emotion data sent from the terminal.

[1255] This analysis identifies factors that the user finds particularly frightening and emotional fluctuations.

[1256] 2. Dynamic scenario update:

[1257] The server dynamically updates the horror scenario based on the analysis results.

[1258] For example, the next ghost will be set to appear from a darker location, and adjustments will be made in real time based on data from the emotion engine.

[1259] Specific use cases

[1260] As users walk around the virtual store and look at the products, they are presented with a horror experience in which a ghost peeks out from behind a shelf. When the user shows signs of surprise and their facial expression changes, the emotion analysis engine recognizes their reaction. In the next scenario, the user is provided with even more surprising elements.

[1261] Prompt Sentence Examples

[1262] "Users were surprised by the scenario where a ghost was peeking out from behind a shelf, so in the next scenario, add a scene where the lights go out and something is hiding under the shelf."

[1263] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1264] Step 1:

[1265] The server receives photos and floor plan data of the home uploaded by the user through a dedicated app. The input data are image files and drawing files provided by the user. To analyze this data, the server uses image analysis software (e.g., OpenCV) to identify the room layout and furniture placement. The output is room layout information and furniture position information.

[1266] Step 2:

[1267] The server generates ghosts and horror scenarios based on the analysis results and emotional data obtained from the user's emotion engine. The input data is the layout information of the residence and emotional data. Using a generative AI model, this data is analyzed to generate optimal ghosts and scenarios. The output is details of ghosts and horror scenarios that are suitable for the user's residence environment.

[1268] Step 3:

[1269] The server sends the generated ghost and scenario information to the user's device. The input data is detailed information about the ghost and scenario, and the output is data sent to the device. The server synchronizes the information using a communication protocol (e.g., HTTP).

[1270] Step 4:

[1271] The device uses cameras and sensors to scan the user's living environment in real time. The input data is on-site video and environmental data acquired by the cameras and sensors. The output data is real-time information about the room's brightness, furniture layout, and the user's location.

[1272] Step 5:

[1273] The device displays ghosts superimposed on the user's real environment based on ghost and scenario information sent from the server. The input data is detailed information about ghosts and scenarios sent from the server, as well as real-time environmental data. The output is ghosts and horror effects superimposed on the user's real environment.

[1274] Step 6:

[1275] The device uses a camera and microphone to monitor and collect the user's actions and reactions in real time. The input data is the user's audio and video information. The output is reaction data such as the user's facial expressions, voice, and movements.

[1276] Step 7:

[1277] The device analyzes the user's emotional data using an emotion engine. The input data is the user's reaction data, and the output is the analyzed user's emotional state. The emotion engine uses facial expression recognition and voice analysis algorithms.

[1278] Step 8:

[1279] The server dynamically updates the horror scenario based on the user's reaction data and emotional data sent from the device. The input data is the user's reaction data and emotional state data. The output is detailed information about the new horror scenario. A new optimal scenario is generated using a generative AI model.

[1280] Step 9:

[1281] The server then sends the updated new scenario information to the device again. The input data is the dynamically adjusted horror scenario details, and the output is the data sent to the device. This information is displayed appropriately on the device, providing the user with a continuous horror experience.

[1282] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1283] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1284] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1285] [Fourth embodiment]

[1286] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1287] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1288] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1289] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1290] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1291] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1292] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1293] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1294] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1295] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1296] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1297] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1298] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1299] The present invention provides an interactive horror experience by utilizing environmental data of the user's home. The program and processing of this system will be described in detail below.

[1300] Server Operation

[1301] Receiving and analyzing environmental data

[1302] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[1303] Generating ghosts and horror scenarios

[1304] The server generates ghost and horror scenarios that best fit the user's environment based on the analyzed residential information. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[1305] Sending scenario information

[1306] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[1307] Device behavior

[1308] environmental awareness

[1309] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[1310] Displaying ghosts and running scenarios

[1311] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[1312] User behavior monitoring

[1313] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the device records the voice and movements and sends them to the server. This data is important for showing when the user felt particularly scared.

[1314] Feedback and scenario updates

[1315] Analysis of user responses and scenario updates

[1316] The server analyzes the user's reaction data sent from the device and identifies the elements that frighten the user. For example, if the user is particularly afraid of the dark, the next ghost displayed will be set to appear from a darker location. In this way, the scenario is dynamically updated, aiming to maximize the sense of fear.

[1317] Providing a continuous horror experience

[1318] The updated new scenario information is then sent back to the device, which receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[1319] Specific examples

[1320] For example, let's say a user starts an experience in their living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghostly hand at the appropriate time. If the user jumps in surprise at this effect, the video and audio of the user's reaction are recorded and sent to the server. The server adjusts the next scenario based on this reaction, continuously evolving the terrifying experience.

[1321] As described above, the present invention is a system that dynamically links with the user's home environment to provide a realistic and immersive horror experience.

[1322] The processing flow will be explained below.

[1323] Step 1:

[1324] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[1325] Step 2:

[1326] Device: Sends uploaded photos and floor plan data to the server.

[1327] Step 3:

[1328] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[1329] Step 4:

[1330] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[1331] Step 5:

[1332] Server: Sends the generated ghosts and scenario information to the user's device.

[1333] Step 6:

[1334] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[1335] Step 7:

[1336] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[1337] Step 8:

[1338] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or gets startled.

[1339] Step 9:

[1340] Terminal: Sends user reaction data to the server.

[1341] Step 10:

[1342] Server: Analyzes the user's reaction data and identifies the elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[1343] Step 11:

[1344] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost may appear from a darker location.

[1345] Step 12:

[1346] Server: Sends updated scenario information to the device.

[1347] Step 13:

[1348] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[1349] In this way, the present invention provides a dynamic and individually optimized horror experience based on the user's living environment.

[1350] Example 1

[1351] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1352] Conventional horror experience systems have had the problem of being unable to provide an immersive experience, as it is difficult to provide a frightening experience that corresponds to the user's individual living environment and behavior. Furthermore, it is difficult to analyze the user's reactions in real time and dynamically update the scenario. This creates the problem of users becoming bored.

[1353] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1354] In this invention, the server includes means for receiving and analyzing environmental information about the user's residence, means for generating a virtual image and a horror experience scenario suited to the user's residence based on the analysis results, and means for transmitting the generated virtual image and scenario information to the information terminal. This makes it possible to provide a horror experience suited to the user's individual environment. The information terminal also includes means for detecting the user's residence environment and user behavior and displaying a virtual image at an appropriate time and place, means for monitoring and collecting user reaction data, and means for analyzing the collected reaction data and dynamically updating the horror experience scenario. This makes it possible to individually optimize the scenario based on the user's reaction and provide a sustained and immersive horror experience.

[1355] 1. "User's place of residence" refers to the place where the user lives on a daily basis, including the environment such as a house or apartment.

[1356] 2. "Environmental information" refers to information about the user's place of residence, including data such as room layout, furniture arrangement, brightness, temperature, and sound.

[1357] 3. "Virtual images" refer to images or visual effects that do not exist in reality but are visually displayed, including horror effects such as ghosts.

[1358] 4. "Horror experience scenario" refers to a series of actions or storylines designed to instill fear in the user.

[1359] 5. "Information terminal" refers to a device used by a user, including a smartphone, tablet, computer, etc.

[1360] 6. “Analysis” means the process or method of examining collected data in detail and extracting necessary information.

[1361] 7. "Detection" refers to sensing the environment or user behavior using devices such as sensors and cameras.

[1362] 8. "Monitoring" means continuously observing user behavior and reactions and obtaining necessary data.

[1363] 9. "Collection" refers to the compilation and retention of data obtained through surveillance.

[1364] 10. "Dynamic updates" refers to changing the scenario or presentation in real time according to the situation.

[1365] 11. "Individual optimization" means providing optimal content tailored to the characteristics of each user based on collected data.

[1366] 12. “Generative AI Model” means a computational model that uses artificial intelligence techniques to generate new data or content.

[1367] The present invention is a system that provides an interactive horror experience by utilizing environmental information of a user's residence. This system realizes an immersive horror experience in real time using a terminal, a server, and cameras and sensors for monitoring the user's actions and reactions. Specific embodiments of this system are described below.

[1368] Server Operation

[1369] Receiving and analyzing environmental information

[1370] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app. After receiving the data, the server analyzes this data to determine the room layout and furniture placement information. For example, if a sofa or bookshelf is located in the living room, it will identify their locations. Image recognition algorithms are used for the analysis.

[1371] Generating ghosts and horror scenarios

[1372] Based on the analyzed residential information, the server generates virtual images and horror scenarios that are optimal for the user's environment. The generative AI model is used to devise scenarios and determine the timing and content of the scenes. For example, it can generate scenes in which a ghostly hand reaches out from under a sofa or a closet door opens on its own.

[1373] Sending scenario information

[1374] The generated virtual images and horror scenarios are sent from the server to the information terminal, where data is communicated and synchronized in real time to ensure a smooth experience for the user.

[1375] Device behavior

[1376] environmental awareness

[1377] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the room's brightness, furniture layout, and the user's location, and sends the information to the server. For example, the device may detect that the user is near a bookshelf.

[1378] Displaying ghosts and running scenarios

[1379] Based on the scenario information sent from the server, the device displays a virtual image superimposed on the user's real environment. For example, if the user is sitting on a sofa, a ghost's hand will appear to reach out from under the sofa.

[1380] User behavior monitoring

[1381] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user screams or gets startled, the voice and movement are recorded and sent to the server. This allows the device to understand when the user is feeling particularly scared.

[1382] Feedback and scenario updates

[1383] Analysis of user responses and scenario updates

[1384] The server analyzes the user's reaction data sent from the device and identifies the elements of fear that should be enhanced in the next scenario. For example, if the user has a strong fear of the dark, the server will set the next scenario to have ghosts appear from darker places.

[1385] Providing a continuous horror experience

[1386] The server then sends the updated scenario information back to the device, which then receives it and provides a new horror experience, ensuring that users always receive a fresh, individually optimized horror experience.

[1387] Specific examples

[1388] 1. The user begins the experience in their living room

[1389] The server analyzes photos of the living room and identifies the location of the bookshelf and sofa.

[1390] Based on this information, the server generates a scenario in which a ghostly hand reaches out from under the sofa and sends it to the device.

[1391] 2. Device-based environment recognition and scenario execution

[1392] The device uses cameras and sensors to scan the user's living environment and detects when the user approaches the sofa.

[1393] A ghostly hand appears at the appropriate time to create a frightening effect.

[1394] 3. User response and data transmission

[1395] When the user screams in surprise, their voice and movements are recorded by the terminal.

[1396] The terminal transmits this reaction data to the server.

[1397] 4. Feedback analysis and scenario update

[1398] The server analyzes the user's reaction data and identifies areas that need to be strengthened in the next scenario. For example, if darkness causes fear, the next performance could have a virtual image appear from a dark place.

[1399] 5. Submit new scenarios and continue the experience

[1400] The server then sends the updated scenario back to the device, which receives it and provides a new horror experience.

[1401] Prompt Sentence Examples

[1402] "Generate a scenario where something crawls out from under the sofa the moment the user enters the living room. Think of a realistic effect that will surprise the user."

[1403] "Create a scenario where a ghost appears from the shadow of the bookshelf when the user goes to get a book. Set the timing to make the user feel scared."

[1404] "I thought of an eerie sound playing in the dark, and when the user heads in the direction of the sound, a ghost will appear."

[1405] As described above, the present invention dynamically links with the user's home environment to provide an individually optimized horror experience.

[1406] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1407] Program processing flow

[1408] Server Operation

[1409] Step 1:

[1410] The server receives photos and floor plan data of the residence uploaded by the user through a dedicated app.

[1411] Input: Photo and floor plan data of the residence submitted by the user.

[1412] Output: Received residence location data.

[1413] Specific operation: Use the file receiving module to obtain image data from the user.

[1414] Step 2:

[1415] The server inputs the received data into an image recognition algorithm for analysis.

[1416] Input: Received residential location photo and floor plan data.

[1417] Output: Analyzed room layout and furniture placement information.

[1418] What it does: Uses image recognition software to identify furniture locations and room layouts.

[1419] Step 3:

[1420] The server uses a generative AI model to generate virtual images and horror experience scenarios based on the analysis results.

[1421] Input: Room layout and furniture placement information.

[1422] Output: Generated virtual images and frightening scenarios.

[1423] Specific operation: A prompt sentence is input into the generative AI model to generate a scenario. For example, it generates a scenario in which a ghost's hand reaches out from under the sofa.

[1424] Step 4:

[1425] The server transmits the generated virtual image and scenario information to the user's terminal.

[1426] Input: Generated virtual image and scenario information.

[1427] Output: Scenario data sent to the user's information terminal.

[1428] Specific operation: The scenario data is transmitted to the user terminal using the data transmission module.

[1429] Device behavior

[1430] Step 5:

[1431] The device uses cameras and sensors to scan the user's living environment in real time and transmits the data to a server.

[1432] Input: Real-world environment data acquired by cameras and sensors.

[1433] Output: The environment data sent to the server.

[1434] Specific operation: The environmental recognition module is used to detect the room brightness, furniture arrangement, and user position in real time.

[1435] Step 6:

[1436] Based on the scenario information sent from the server, the terminal displays a virtual image superimposed on the user's real environment.

[1437] Input: Scenario information sent from the server.

[1438] Output: Displayed virtual image and frightening experience scenario.

[1439] Specific operation: Using the AR (Augmented Reality) module, a ghost hand appears from under the sofa when the user approaches it.

[1440] Step 7:

[1441] The device uses a camera and microphone to monitor the user's actions and reactions in real time and transmits the data to a server.

[1442] Input: User behavior and voice data captured by camera and microphone.

[1443] Output: User response data sent to the server.

[1444] Specific actions: Use the monitoring module to record the user's voice and actions such as shouting or being surprised.

[1445] Feedback and scenario updates

[1446] Step 8:

[1447] The server analyzes the user's reaction data sent from the device and identifies the fear elements that should be strengthened in the next scenario.

[1448] Input: User reaction data sent from the device.

[1449] Output: Scenario with new and enhanced horror elements based on analysis.

[1450] Specific behavior: Use the data analysis module to identify the moments and elements that made users feel most scared.

[1451] Step 9:

[1452] The server generates new scenarios and sends them to the user's device, continually updating the terrifying experience.

[1453] Input: User's reaction data analysis results.

[1454] Output: New scenario data.

[1455] Specific operation: New conditions are input into the generative AI model to generate scenarios with enhanced fear elements, such as "a scenario in which a ghost appears from a dark place."

[1456] As described above, this system allows the server and device to cooperate to provide users with an individually optimized interactive horror experience.

[1457] (Application example 1)

[1458] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1459] Conventional horror experience systems do not adequately generate horror scenarios that are dependent on the user's real-world environment, nor do they dynamically update the scenarios based on the user's real-time actions and reactions. As a result, there are issues with the user's sense of fear and immersion being limited. In addition, it has been difficult to provide a horror experience through interaction in a virtual real space.

[1460] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1461] In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and a horror scenario appropriate for the user's residence based on the analysis results, means for transmitting the generated ghost and scenario information to the terminal, means for recognizing the user's residence environment and user behavior in the terminal and displaying a ghost at an appropriate time and place, means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for causing a ghost's hand to appear from a specific object when the user approaches the object in the virtual real space, and means for generating a darkness effect and causing a ghost to appear in that location when the user performs a certain action in a specific area in the virtual real space, thereby providing an interactive and immersive horror experience in both the user's real environment and the virtual real space.

[1462] "Environmental data in the user's residence" is data including the layout, furniture arrangement, lighting conditions, and other related information within the user's residence.

[1463] "Ghost and Horror Scenarios" are scenes of mysterious phenomena and terrifying experiences that are generated based on the user's interactions in their living environment and virtual real space.

[1464] "Terminals" are devices such as smartphones, smart glasses, and head-mounted displays that allow users to experience horror scenarios visually and aurally.

[1465] "Means for recognizing user behavior" refers to a function that detects information such as the user's position, movements, voice, and facial expressions in real time.

[1466] "User reaction data" refers to data including behavior, vocal responses, and physiological responses such as heart rate that a user exhibits in response to a horror scenario.

[1467] "Means for dynamically updating horror scenarios" is a function that analyzes collected user reaction data and changes the content and presentation of the scenario to provide the user with the optimal horror experience.

[1468] A "virtual real space" is a virtual reality environment, a digital space designed for users to experience.

[1469] The "effect of a ghostly hand appearing from the object" is an effect in which, when a user approaches a specific virtual object, a mysterious phenomenon (for example, a ghostly hand) visually occurs from the object.

[1470] "Means for creating a darkness effect and making a ghost appear in that location" refers to a production in which, when a user performs a specific action in a specific area of ​​the virtual real space, the lights in that area are dimmed and a ghost appears.

[1471] This invention is a system that utilizes data on a user's home environment to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[1472] The server has a means for receiving and analyzing environmental data of the user's residence, including photos and floor plan data of the user's residence. Based on this data, the server analyzes the room layout and furniture arrangement to generate specific ghost and horror scenarios.

[1473] Based on the analysis results, the server has the means to generate ghosts and horror scenarios that are appropriate for the user's residence. For example, if the user has a sofa in their living room, the server will generate a scenario in which a ghost's hand reaches out from under the sofa. This generated ghost and scenario information is sent to the device via a dedicated application.

[1474] The device has a means of recognizing the user's home environment and behavior. The device is equipped with a camera and sensors, and can detect the user's location and behavior in real time. For example, when the user approaches a specific location, that information is sent to the server.

[1475] The device also has a means to display ghosts at the appropriate time and place. Based on the scenario information sent from the server, the device can display ghosts superimposed on the user's real environment. For example, when the user is sitting on a sofa, the device will display an effect in which a ghost's hand reaches out from under the sofa.

[1476] Additionally, the device has a means of monitoring and collecting user reaction data. It uses a camera and microphone to record the user's actions and reactions and transmits the data to a server. This reaction data includes when the user shouts or shows surprise.

[1477] The server analyzes the collected reaction data and dynamically updates the horror scenario. It identifies the elements that frighten the user most and reflects this in the next scenario. For example, if the user is very scared of the dark, the next ghost that appears will be set to appear in a darker location.

[1478] Furthermore, this system also includes a means for creating an effect in which a ghostly hand appears from a specific object when the user approaches the object in the virtual real space. For example, it is possible to create a scenario in which a ghostly hand reaches out from a specific product shelf in a virtual store when the user approaches the product.

[1479] The virtual reality system also has a means for generating a darkness effect and making a ghost appear in a specific area in the virtual real space when the user performs a certain action in that area. For example, when the user enters a specific area in the virtual real space, the lighting in that area suddenly becomes dark and a ghost appears.

[1480] As a specific example, when a user approaches a specific product in a virtual store, a ghost's hand may reach out from under the product.Also, when a user approaches a specific area in the virtual store, the lights in that area may suddenly dim and a ghost may appear.

[1481] Examples of prompt sentences include:

[1482] "Create a scenario in a virtual store where ghostly hands reach out from the shelves when the user approaches a particular product."

[1483] "Create a horror scenario where when a user enters a certain area, the area becomes dark and ghosts appear."

[1484] In this way, it is possible to provide a highly interactive and immersive horror experience in both the user's real environment and the virtual real space.

[1485] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1486] Step 1:

[1487] The server receives and analyzes environmental data at the user's residence.

[1488] Input: Photos and floor plan data of the residence uploaded by the user through a dedicated app

[1489] What it does: It receives this data and uses image analysis algorithms to determine the room layout and furniture placement.

[1490] Output: Room layout and furniture placement information

[1491] Step 2:

[1492] Based on the analysis results, the server generates ghosts and horror scenarios that are appropriate for the user's residence.

[1493] Input: Room layout and furniture placement information

[1494] Specific operation: Generate the location and performance details for ghost appearances for each room, for example, creating a scenario in which a ghost's hand reaches out from under the sofa.

[1495] Output: Ghost and horror scenario information

[1496] Step 3:

[1497] The server transmits the generated ghost and scenario information to the terminal.

[1498] Input: Ghost or horror scenario information

[1499] Specific operation: The server packages the information and sends it to the user's device through a dedicated application.

[1500] Output: Scenario information sent to the device

[1501] Step 4:

[1502] The terminal recognizes the user's home environment and the user's behavior.

[1503] Input: Received scenario information, real-time data from the device's camera and sensors

[1504] Specific operation: Using the device's camera and sensors, the device detects the user's location, movements, room brightness, etc. in real time and analyzes them.

[1505] Output: User's current location and movement information

[1506] Step 5:

[1507] The device will display the ghost at the appropriate time and place.

[1508] Input: User's current location and movement information, received scenario information

[1509] Specific behavior: When the user approaches a location specified in the scenario, a ghostly hand or other effect will be displayed. For example, a ghostly hand will appear reaching out from under the sofa.

[1510] Output: The displayed ghost scene

[1511] Step 6:

[1512] The terminal monitors and collects the user's reaction data.

[1513] Input: User reaction data captured by camera and microphone (e.g., vocal responses, facial expressions)

[1514] What it does: The device uses its camera and microphone to record the user's reactions and sends the data to a server.

[1515] Output: Collected user response data

[1516] Step 7:

[1517] The server analyzes the collected reaction data and dynamically updates the horror scenario.

[1518] Input: Collected user response data

[1519] Specific behavior: Identify the timing and elements that made the user feel particularly scared, and adjust the scenario accordingly. For example, if the user is particularly scared of the dark, the next scenario will have ghosts appear from even darker locations.

[1520] Output: Updated horror scenario

[1521] Step 8:

[1522] The device will then display ghosts at the appropriate times and locations based on the new horror scenario, providing a continuous horror experience.

[1523] Input: Updated horror scenario

[1524] Specific behavior: The device receives the new scenario and displays ghosts again based on the user's actions, for example, adding new ghost effects to dark areas.

[1525] Output: The new ghost scene displayed

[1526] In this way, by linking each step together, it is possible to provide an interactive and immersive horror experience in both the user's home environment and the virtual real world.

[1527] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1528] The present invention is a system that utilizes environmental data and emotional data from a user's home to provide an interactive horror experience. The program and processing of this system will be described in detail below.

[1529] Server Operation

[1530] Receiving and analyzing environmental data

[1531] The server receives photos and floor plan data uploaded by users through a dedicated app, analyzes the received data, and determines the layout of the rooms and furniture. For example, if there is a sofa or bookshelf in the living room, it identifies their locations.

[1532] Generating ghosts and horror scenarios

[1533] The server generates ghost and horror scenarios that are optimal for the user's environment based on the analyzed residential information and emotion data obtained from the emotion engine. For example, scenarios such as a ghost's hand reaching out from under the sofa or a closet door opening on its own may be prepared. The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the scene.

[1534] Sending scenario information

[1535] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[1536] Device behavior

[1537] environmental awareness

[1538] The device uses cameras and sensors to scan the user's living environment in real time, recognizing the brightness of the room, the layout of furniture, and the user's location. For example, if the user is near a bookshelf, the device sends that location information to the server.

[1539] Displaying ghosts and running scenarios

[1540] The device displays ghosts superimposed on the user's real-world environment based on ghost and scenario information sent from the server. For example, when the user is sitting on a sofa, a ghost's hand is displayed reaching out from under the sofa in real time.

[1541] User behavior and sentiment monitoring

[1542] The device uses a camera and microphone to monitor the user's actions and reactions in real time. If the user shouts or shows surprise, the device records their voice and movements, and the emotion engine analyzes the user's emotions from their facial expressions and voice. This data is then sent to the server.

[1543] Feedback and scenario updates

[1544] User response and emotion analysis

[1545] The server analyzes the user's reaction and emotional data sent from the device. This analysis identifies factors that frighten the user and changes in their emotions, such as fear of the dark or being startled by sudden movements.

[1546] Dynamic scenario updates

[1547] The server dynamically updates the horror scenario based on the analysis results. For example, it may set the next ghost to appear in a darker location, and adjusts in real time based on the emotion engine data. This provides an optimal horror experience according to the user's reactions and emotions.

[1548] Providing a continuous horror experience

[1549] The updated new scenario information is then sent back to the device, which receives it and provides the user with a new horror experience, ensuring that the user always receives a fresh, individually optimized horror experience.

[1550] Specific examples

[1551] For example, let's say a user starts an experience in a living room. The server analyzes photos of the living room and identifies the locations of the bookshelf and sofa. Using this information, the server generates a scenario in which a ghost's hand reaches out from under the sofa and sends it to the device. The device uses its camera to recognize the user approaching the sofa and displays the ghost's hand at the appropriate time. The user jumps in surprise at this effect, and the device analyzes their facial expression using its emotion engine. The server adjusts the next scenario based on the analysis results and displays a scenario in which a ghost peeks out from behind the bookshelf.

[1552] In summary, the present invention is a system that provides a realistic and immersive horror experience based on the user's living environment and emotions.

[1553] The processing flow will be explained below.

[1554] Step 1:

[1555] User: Downloads the dedicated app, takes photos of each room in their home, and uploads them to the app.

[1556] Step 2:

[1557] Device: Sends uploaded photos and floor plan data to the server.

[1558] Step 3:

[1559] Server: Analyzes the received photos and floor plan data to determine the room layout and furniture placement. For example, it recognizes that a sofa and bookshelf are placed in the living room.

[1560] Step 4:

[1561] Server: Based on the analysis results, it generates ghost and horror scenarios that are best suited to the user's home. For example, it generates a scenario in which a ghost hand appears from under the sofa.

[1562] Step 5:

[1563] Server: Sends the generated ghosts and scenario information to the user's device.

[1564] Step 6:

[1565] Device: Based on the received ghost and horror scenario information, the device uses cameras and sensors to scan the user's living environment in real time and recognizes the furniture layout and the user's location.

[1566] Step 7:

[1567] Device: Based on real-time environmental data, ghosts are displayed at the appropriate time and place. For example, if the user is sitting on a sofa, a ghost's hand will reach out from under the sofa.

[1568] Step 8:

[1569] Device: Uses cameras and microphones to monitor user actions and reactions in real time, for example, recording audio and video when the user shouts or gets startled.

[1570] Step 9:

[1571] Device: Using the user's emotion engine, emotions such as fear and surprise are analyzed from the user's facial expressions and voice.

[1572] Step 10:

[1573] Terminal: Emotion data analyzed by the emotion engine is sent to the server.

[1574] Step 11:

[1575] Server: Analyzes the user's reaction and emotional data sent to identify elements that the user finds particularly frightening, such as being afraid of the dark or being startled by sudden movements.

[1576] Step 12:

[1577] Server: Based on the analysis results, the horror scenario is dynamically updated. For example, the next ghost to appear will be set to appear from a darker location.

[1578] Step 13:

[1579] Server: Sends updated scenario information to the device.

[1580] Step 14:

[1581] Terminal: New ghosts and scenarios are reflected in the residential environment, providing the user with a continuous horror experience. For example, the next scenario will show a ghost peeking out from behind a bookshelf.

[1582] As described above, the present invention is a system that dynamically generates a horror scenario based on the user's living environment and uses an emotion engine to analyze the user's emotions in real time, thereby providing an individually optimized interactive horror experience.

[1583] Example 2

[1584] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1585] Conventional horror experience systems have difficulty reflecting the user's living environment and individual emotional state in real time, making it impossible to provide a unique and immersive horror experience for the user. Furthermore, with static scenarios, the sense of fear fades after a scene is experienced once, making it difficult to maintain sustained excitement.

[1586] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data related to the user's living space, means for generating a virtual character and a horror scenario suited to the user's living space based on the analysis results, and means for transmitting the generated character and scenario information to the communication terminal. This enables a highly realistic and individually optimized horror experience tailored to the user's environment.

[1587] "User" refers to an individual who uses this system.

[1588] "Living space" refers to the home or room where a user lives their daily life.

[1589] "Environmental data" refers to information about the user's living space, such as photographs, floor plans, and furniture layout.

[1590] "Analysis" refers to a series of processes that analyze information based on received environmental data and identify the room layout and furniture placement.

[1591] "Virtual character" refers to a fear-inducing artificial character, such as a ghost, that appears in the user's living space.

[1592] A "horror scenario" refers to a sequence or performance designed to make the user feel fear.

[1593] "Communication terminal" refers to electronic devices used by users, such as smartphones and tablets.

[1594] "Monitoring" refers to the continuous observation of a user's behavior and reactions.

[1595] "Collect" refers to recording and storing user response data.

[1596] "Dynamic updates" refers to changing and adjusting scenarios and information in real time.

[1597] The present invention provides a system for providing an interactive horror experience using environmental data and emotional data in a user's living space. Specific embodiments of the system will be described in detail below.

[1598] Server Operation

[1599] The server first receives photos and floor plan data uploaded by the user through a dedicated app. This environmental data is sent to the server in JSON format. The server then uses a machine learning model (e.g., YOLOv5) to analyze the furniture placement and room layout from the received image data. The analysis results are stored in a database.

[1600] Next, emotional data is collected. The server uses an emotion analysis API (e.g., Azure Emotion API) to analyze the user's past reaction data. This allows the server to evaluate the situations in which the user feels particularly scared.

[1601] Based on the analyzed environmental data and emotional data, the server uses a generative AI model (e.g., GPT-4) to generate a horror scenario. The prompt includes instructions such as, "Please generate a scenario that will frighten the user." The generated scenario includes detailed information about the locations and timing of virtual character appearances. This scenario information is then encoded again into JSON format and sent to the user's communication device.

[1602] Device behavior

[1603] The device is equipped with a camera and sensors and scans the user's living environment in real time. The scan data is stored in the cloud and sent to the server as needed. The device also displays a virtual character in the user's real environment based on the horror scenario received from the server. Specifically, the device uses an augmented reality (AR) framework (e.g., ARKit) to overlay the virtual character on the user's camera image.

[1604] The device also uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to the server.

[1605] Feedback and scenario updates

[1606] The server analyzes the user's reaction data sent from the device and identifies elements that are particularly likely to cause fear. Based on the analysis results, a new prompt is given to the generative AI model to generate the next scenario. For example, the prompt might be, "Please generate a situation that makes the user feel even more scared."

[1607] The new scenario is then sent back to the device, which receives it and provides the user with a new horror experience. This feedback loop ensures that the user always receives a fresh, individually optimized horror experience.

[1608] Specific examples

[1609] For example, if a user wants to start the experience in their living room, they first launch a dedicated app and upload a photo of their living room to the server. The server analyzes the received data and identifies the locations of the bookshelf and sofa. The generative AI model then generates a scenario by providing a prompt: "Generate a scenario in which a ghost's hand reaches out from under the sofa." Based on this scenario, the device recognizes the user approaching the sofa and displays a virtual character. If the user jumps in surprise, their facial expression and voice are recorded and sent to the server. The server then analyzes this reaction data and generates the next scenario using a new prompt: "Generate a scenario in which a ghost peeks out from behind the bookshelf."

[1610] The above is a specific example of an embodiment of the present invention. This system allows users to enjoy a highly realistic and individually optimized horror experience.

[1611] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1612] Step 1:

[1613] The server receives photos and floor plan data of the living space uploaded by the user through a dedicated app. This input data includes images in JPEG and PNG format and floor plans in JSON format. The received data is stored in a database and prepared for analysis.

[1614] Step 2:

[1615] The server analyzes the received living space data. Specifically, it uses a machine learning model (e.g., YOLOv5) to identify furniture placement and room layout from the images. The analysis results are output in JSON format and stored in a database. This allows the location and type of each piece of furniture in the room to be identified.

[1616] Step 3:

[1617] The server uses an emotion analysis API (e.g., Azure Emotion API) to collect user emotion data. The input data includes past user reaction data and audio and video data. The emotion analysis engine analyzes this data and evaluates the situations in which the user feels particularly scared. The evaluation results are output in JSON format and saved as analysis results.

[1618] Step 4:

[1619] The server uses a generative AI model (e.g., GPT-4) to generate a horror scenario based on the analyzed environmental and emotional data. The prompt is "Please generate a scenario that will frighten the user." The environmental and emotional data are provided as input to the generative AI model, and the generated horror scenario is obtained as output. This scenario is output in JSON format, detailing the locations and timing of character appearances.

[1620] Step 5:

[1621] The server sends the generated horror scenario to the user's communication device. The scenario information is encoded in JSON format and sent to the device via an HTTP POST request. The device analyzes the received data and makes preparations.

[1622] Step 6:

[1623] The device uses cameras and sensors to scan the user's living environment in real time, and uses LiDAR sensors and RGB cameras to perform 3D mapping to obtain detailed layout data of the room the user is in. The scanned data is then stored in the cloud in real time.

[1624] Step 7:

[1625] The device displays a virtual character in the user's real-world environment based on the horror scenario received from the server. Using an augmented reality (AR) framework (e.g., ARKit), the virtual character is overlaid on the user's camera image. Information about the character's appearance location and the content of the presentation is processed and displayed in real time.

[1626] Step 8:

[1627] The device uses a camera and microphone to monitor the user's actions and reactions. If the user is surprised or screams, audio and video data is collected in real time and sent to a server.

[1628] Step 9:

[1629] The server analyzes the user's reaction data sent from the device and identifies the elements that cause particular fear. It uses an emotion analysis API to analyze the reaction data and saves the analysis results in JSON format. This allows the user's fear triggers to be identified.

[1630] Step 10:

[1631] The server generates the next horror scenario by providing a new prompt to the generative AI model based on the analysis results. The prompt uses "Please generate a situation that will make the user feel even more scared." The server provides the analysis results and environmental data as input data, and obtains a new scenario as output.

[1632] Step 11:

[1633] The server then sends the generated new scenario to the communication terminal again. The scenario information is encoded in JSON format and sent to the terminal via an HTTP POST request, providing the user with a new horror experience.

[1634] The above are the specific processing steps of the program of this system.

[1635] (Application example 2)

[1636] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1637] In recent years, with the spread of virtual stores, there has been a demand for improved user experience. However, conventional virtual stores lack a system that can analyze users' emotions and reactions in real time and provide an optimal entertainment experience accordingly. The lack of such a system prevents users from getting an immersive experience, which reduces the appeal of virtual stores.

[1638] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing environmental data of the user's residence, means for generating ghosts and horror scenarios appropriate for the user's residence based on the analysis results, and means for transmitting the generated ghosts and scenario information to the terminal. This makes it possible to analyze the user's emotional data in real time and display ghosts at appropriate times and locations in the virtual store. In addition, by including means for monitoring and collecting user reaction data, means for analyzing the collected reaction data and dynamically updating the horror scenario, means for displaying ghosts at appropriate times and locations in the virtual environment as the user walks around the virtual store environment, and means for analyzing the user's emotional data in real time and using it to generate new horror scenarios, it is possible to provide an interactive horror experience that responds to the user's actions and emotions.

[1639] "Environmental data in the user's home" is data collected through cameras and sensors, including information on the layout, objects, brightness, temperature, and other aspects of the user's home.

[1640] "Means for analysis" refers to a processing device or software that uses algorithms or artificial intelligence to analyze collected environmental data and emotional data and extract specific information.

[1641] "Ghosts and horror scenarios" refer to virtual supernatural phenomena and the circumstances under which they appear that are designed to frighten users.

[1642] The "means for generating" is a processing device or software that creates new ghosts and horror scenarios based on the analysis results and emotional data.

[1643] The "transmitting means" is a communication device or software for transmitting the generated ghost and horror scenario information from the server to the user's terminal.

[1644] "Terminal" refers to electronic devices used by users, such as smartphones, tablets, and head-mounted displays (HMDs).

[1645] A "means for recognizing user behavior" is a device or software that uses a camera or sensor to detect the user's movements and location and analyzes that information.

[1646] The "means for displaying ghosts at the appropriate time and place" refers to a processing device or software that displays ghosts or horror scenarios at the appropriate time and place based on the user's behavior and location information in order to provide an optimal frightening experience.

[1647] "User reaction data" is data that records the user's reactions to the horror scenario, such as facial expressions, voice, and movements.

[1648] "Monitoring and collecting means" refers to devices or software for monitoring user responses in real time and collecting that data.

[1649] The "dynamic updating means" is a processing device or software for changing or evolving the current horror scenario based on collected user reaction data.

[1650] A "virtual store environment" is a virtual store interior space that is displayed on a device used by a user.

[1651] "Emotion data" is data that represents the user's emotional state analyzed from facial expressions, voice, body movements, and the like.

[1652] In this invention, the following hardware and software are used to build a system that utilizes the user's living environment and emotional data to provide an interactive horror experience in real time.

[1653] Server Operation

[1654] 1. Receiving and analyzing environmental data:

[1655] The server first receives photos and floor plan data of the home uploaded by the user through a dedicated app.

[1656] The received data is analyzed using image analysis software (e.g., OpenCV) to determine the room layout and furniture placement.

[1657] 2. Ghost and Horror Scenarios Generation:

[1658] The server generates ghost and horror scenarios that are optimal for the user's environment based on the detected residential information and emotion data obtained from the emotion engine.

[1659] The generated scenario includes the timing and location of the ghost's appearance, as well as the details of the performance.

[1660] 3. Send scenario information:

[1661] The generated ghosts and horror scenarios are sent from the server to the user's device. The server communicates and synchronizes data so that this information can be displayed appropriately on the device.

[1662] Device behavior

[1663] 1. Environmental awareness:

[1664] The device uses cameras and sensors (e.g., a smartphone's built-in camera and accelerometer) to scan the user's living environment in real time.

[1665] This allows the system to recognize the brightness of the room, the layout of furniture, and the user's position.

[1666] 2. Displaying ghosts and running scenarios:

[1667] The terminal displays the ghost superimposed on the user's real environment based on the ghost and scenario information sent from the server.

[1668] For example, when a user is sitting on a sofa, a ghost's hand is displayed in real time from under the sofa.

[1669] 3. User behavior and sentiment monitoring:

[1670] The device uses a camera and microphone to monitor the user's actions and reactions in real time.

[1671] When a user screams or gets surprised, their voice and movements are recorded, and the emotion engine analyzes the user's emotions from their facial expressions and voice.

[1672] Feedback and scenario updates

[1673] 1. User response and sentiment analysis:

[1674] The server analyzes the user's reaction data and emotion data sent from the terminal.

[1675] This analysis identifies factors that the user finds particularly frightening and emotional fluctuations.

[1676] 2. Dynamic scenario update:

[1677] The server dynamically updates the horror scenario based on the analysis results.

[1678] For example, the next ghost will be set to appear from a darker location, and adjustments will be made in real time based on data from the emotion engine.

[1679] Specific use cases

[1680] As users walk around the virtual store and look at the products, they are presented with a horror experience in which a ghost peeks out from behind a shelf. When the user shows signs of surprise and their facial expression changes, the emotion analysis engine recognizes their reaction. In the next scenario, the user is provided with even more surprising elements.

[1681] Prompt Sentence Examples

[1682] "Users were surprised by the scenario where a ghost was peeking out from behind a shelf, so in the next scenario, add a scene where the lights go out and something is hiding under the shelf."

[1683] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1684] Step 1:

[1685] The server receives photos and floor plan data of the home uploaded by the user through a dedicated app. The input data are image files and drawing files provided by the user. To analyze this data, the server uses image analysis software (e.g., OpenCV) to identify the room layout and furniture placement. The output is room layout information and furniture position information.

[1686] Step 2:

[1687] The server generates ghosts and horror scenarios based on the analysis results and emotional data obtained from the user's emotion engine. The input data is the layout information of the residence and emotional data. Using a generative AI model, this data is analyzed to generate optimal ghosts and scenarios. The output is details of ghosts and horror scenarios that are suitable for the user's residence environment.

[1688] Step 3:

[1689] The server sends the generated ghost and scenario information to the user's device. The input data is detailed information about the ghost and scenario, and the output is data sent to the device. The server synchronizes the information using a communication protocol (e.g., HTTP).

[1690] Step 4:

[1691] The device uses cameras and sensors to scan the user's living environment in real time. The input data is on-site video and environmental data acquired by the cameras and sensors. The output data is real-time information about the room's brightness, furniture layout, and the user's location.

[1692] Step 5:

[1693] The device displays ghosts superimposed on the user's real environment based on ghost and scenario information sent from the server. The input data is detailed information about ghosts and scenarios sent from the server, as well as real-time environmental data. The output is ghosts and horror effects superimposed on the user's real environment.

[1694] Step 6:

[1695] The device uses a camera and microphone to monitor and collect the user's actions and reactions in real time. The input data is the user's audio and video information. The output is reaction data such as the user's facial expressions, voice, and movements.

[1696] Step 7:

[1697] The device analyzes the user's emotional data using an emotion engine. The input data is the user's reaction data, and the output is the analyzed user's emotional state. The emotion engine uses facial expression recognition and voice analysis algorithms.

[1698] Step 8:

[1699] The server dynamically updates the horror scenario based on the user's reaction data and emotional data sent from the device. The input data is the user's reaction data and emotional state data. The output is detailed information about the new horror scenario. A new optimal scenario is generated using a generative AI model.

[1700] Step 9:

[1701] The server then sends the updated new scenario information to the device again. The input data is the dynamically adjusted horror scenario details, and the output is the data sent to the device. This information is displayed appropriately on the device, providing the user with a continuous horror experience.

[1702] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1703] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1704] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1705] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1706] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1707] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1708] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1709] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1710] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1711] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1712] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1713] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1714] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1715] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1716] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1717] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1718] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1719] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1720] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1721] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1722] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1723] The following is further disclosed regarding the above embodiment.

[1724] (Claim 1)

[1725] means for receiving and analyzing environmental data at a user's residence;

[1726] A means for generating ghosts and horror scenarios suitable for the user's residence based on the analysis results;

[1727] means for transmitting the generated ghost and scenario information to a terminal;

[1728] A means for recognizing the user's living environment and behavior in the terminal and displaying a ghost at an appropriate time and place;

[1729] means for monitoring and collecting user response data;

[1730] A means of analyzing the collected reaction data and dynamically updating the horror scenario;

[1731] A system including:

[1732] (Claim 2)

[1733] The system according to claim 1, further comprising means for analyzing the furniture arrangement and floor plan of the user's residence and determining the location of the ghost to appear and the content of the performance based thereon.

[1734] (Claim 3)

[1735] 10. The system of claim 1, further comprising means for detecting when the user feels particularly frightened by monitoring the user's vocal responses and updating the horror scenario based on that information.

[1736] "Example 1"

[1737] (Claim 1)

[1738] means for receiving and analyzing environmental information at a user's residence;

[1739] A means for generating a virtual image and a frightening experience scenario suitable for the user's place of residence based on the analysis results;

[1740] a means for transmitting the generated virtual image and scenario information to an information terminal;

[1741] a means for detecting the user's living environment and behavior in the information terminal and displaying a virtual image at an appropriate time and place;

[1742] means for monitoring and collecting user response data;

[1743] A means for analyzing the collected reaction data and dynamically updating the fear experience scenario;

[1744] Using generative AI models that individually optimize scenarios based on user behavior and reactions,

[1745] A system including:

[1746] (Claim 2)

[1747] 2. The system according to claim 1, further comprising means for analyzing the furniture arrangement and floor plan of the user's residence and determining the location where the virtual image will appear and the content of the presentation based on the analysis.

[1748] (Claim 3)

[1749] 10. The system of claim 1, further comprising means for detecting when the user feels particularly frightened by monitoring the user's vocal responses, and updating the frightening experience scenario based on that information.

[1750] "Application Example 1"

[1751] (Claim 1)

[1752] means for receiving and analyzing environmental data at a user's residence;

[1753] A means for generating ghosts and horror scenarios suitable for the user's residence based on the analysis results;

[1754] means for transmitting the generated ghost and scenario information to a terminal;

[1755] A means for recognizing the user's living environment and behavior in the terminal and displaying a ghost at an appropriate time and place;

[1756] means for monitoring and collecting user response data;

[1757] A means of analyzing the collected reaction data and dynamically updating the horror scenario;

[1758] a means for producing an effect in which a ghost hand appears from a specific object when the user approaches the object in the virtual real space;

[1759] a means for generating a darkness effect and making a ghost appear in a specific area within the virtual real space when the user performs a certain action;

[1760] A system including:

[1761] (Claim 2)

[1762] The system according to claim 1, further comprising means for analyzing the furniture arrangement and floor plan of the user's residence and determining the location of the ghost to appear and the content of the performance based thereon.

[1763] (Claim 3)

[1764] 10. The system of claim 1, further comprising means for detecting when the user feels particularly frightened by monitoring the user's vocal responses and updating the horror scenario based on that information.

[1765] "Example 2: Combining Emotion Engines"

[1766] (Claim 1)

[1767] a device for receiving and analyzing environmental data relating to a user's living space;

[1768] a device that generates a virtual character and a horror scenario suited to the user's living space based on the analysis results;

[1769] a device for transmitting the generated character and scenario information to a communication terminal;

[1770] a device that recognizes the user's living environment and behavior in the communication terminal and displays a virtual character at an appropriate time and place;

[1771] a device for monitoring and collecting user response data;

[1772] A device that analyzes collected reaction data and dynamically updates fear scenarios;

[1773] A system including:

[1774] (Claim 2)

[1775] 10. The system according to claim 1, further comprising a device that analyzes the layout and layout of furniture in the user's living space and determines the location where the virtual character will appear and the content of the presentation based on the analysis.

[1776] (Claim 3)

[1777] 10. The system of claim 1, further comprising a device that monitors the user's vocal responses to detect when the user feels particularly scared and updates the scaring scenario based on that information.

[1778] "Application example 2 when combining emotion engines"

[1779] (Claim 1)

[1780] means for receiving and analyzing environmental data at a user's residence;

[1781] A means for generating ghosts and horror scenarios suitable for the user's residence based on the analysis results;

[1782] means for transmitting the generated ghost and scenario information to a terminal;

[1783] A means for recognizing the user's living environment and behavior in the terminal and displaying a ghost at an appropriate time and place;

[1784] means for monitoring and collecting user response data;

[1785] A means of analyzing the collected reaction data and dynamically updating the horror scenario;

[1786] means for displaying ghosts at appropriate times and places within the virtual environment as the user walks around the virtual store environment;

[1787] A means for analyzing user emotion data in real time and using the data to generate new horror scenarios;

[1788] A system including:

[1789] (Claim 2)

[1790] The system according to claim 1, further comprising means for analyzing the furniture arrangement and floor plan of the user's residence and determining the location of the ghost to appear and the content of the performance based thereon.

[1791] (Claim 3)

[1792] 10. The system of claim 1, further comprising means for detecting when the user feels particularly frightened by monitoring the user's vocal responses and updating the horror scenario based on that information. [Explanation of symbols]

[1793] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving and analyzing environmental data at a user's residence; A means for generating ghosts and horror scenarios suitable for the user's residence based on the analysis results; means for transmitting the generated ghost and scenario information to a terminal; A means for recognizing the user's living environment and behavior in the terminal and displaying a ghost at an appropriate time and place; means for monitoring and collecting user response data; A means of analyzing the collected reaction data and dynamically updating the horror scenario; A system including:

2. 2. The system according to claim 1, further comprising means for analyzing the furniture arrangement and floor plan of the user's residence and determining the location of the ghost to appear and the content of the performance based thereon.

3. The system of claim 1 further comprising means for detecting when the user feels particularly frightened by monitoring the user's vocal responses and updating the horror scenario based on that information.

Images