system

Abstract

Provide a system. [Solution] A means of obtaining the artist's design; a generating means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby; terminal means for receiving and preparing the generated interactive art data; a sensor means for detecting the movement of passersby; a projection means for dynamically adjusting art data based on data from the sensor and projecting the art data into a public space; A collaborative method for collecting data on local historical elements and popular characters and providing it to the generation method. A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional street art and art in public spaces are static and offer limited interaction with passersby, which means the appeal of the art is not fully realized and it is difficult to revitalize the local area. Furthermore, because the art is one-directional, it is difficult for passersby to actively engage with it, limiting experiential artistic expression. Furthermore, there has been a lack of effective ways to integrate art with local history and culture. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means: A system including a means for acquiring a design drawn by an artist, a generation means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby, a terminal means for receiving and preparing the generated interactive art data, a sensor means for detecting the movements of passersby, a projection means for dynamically adjusting the art data based on the data from the sensors and projecting it into a public place, and a collaboration means for collecting data on local historical elements, popular characters, etc. and providing it to the generation means, thereby enhancing interaction with passersby, providing an attractive art experience, and promoting the revitalization of the local area.

[0006] "Artist" refers to a professional who creates designs for the visual arts.

[0007] "Design" is the visual composition or layout created by an artist.

[0008] "Acquisition" refers to the act of obtaining specific information from a database or other storage medium.

[0009] "Analysis" refers to the process of examining and interpreting acquired design data in detail to extract information for incorporating interactive elements.

[0010] "Passersby" refers to ordinary people moving through public spaces.

[0011] "Movement" refers to the physical movements and gestures of passersby.

[0012] "Interactive elements" refer to parts of art that change dynamically in response to user actions or changes in the environment.

[0013] "Generative means" refers to the mechanism or process used to create interactive elements from a design.

[0014] "Art Data" refers to digital information that integrates design and interactive elements.

[0015] "Terminal means" refers to a device or system that receives and processes art data and prepares it for projection.

[0016] "Sensor means" refers to devices or technologies that detect the movement of passersby or changes in the environment.

[0017] "Projection means" refers to a device or system for projecting adjusted art data onto a physical wall or screen.

[0018] "Collaborative means" refers to the process or method for collecting data such as local historical elements or popular characters and integrating it into art. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists. The system's main components are a server, a terminal, and a user.

[0041] System Overview

[0042] 1. The server acquires and analyzes the artist's design

[0043] The first step the server takes is to retrieve the artist's design from the database. This design is visually represented and includes tags and metadata. The server then asks the generation AI to analyze the retrieved design. Once the generation AI begins its analysis, it extracts each element in the design and adds interactive elements (for example, parts that change color depending on how close passersby are).

[0044] 2. Generating Interactive Art with Generative AI

[0045] The AI ​​analyzes the design received from the server and begins generating interactive art. The generated art has dynamic elements, such as changing color or shape when passersby approach. The generated interactive art data is then returned to the server.

[0046] 3. Data reception by the terminal and preparation for projection

[0047] The server sends the generated interactive art data to the device. The device stores the received art data in a buffer and prepares for projection. At this time, the device checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest art data.

[0048] 4. Detecting and Processing User Movements

[0049] When a user (passerby) moves through a public space, sensors detect their movements. The data measured by the sensors is sent to the device in real time. The sensors accurately measure the passerby's distance and type of movement (e.g., waving, jumping, etc.). The device receives this data and dynamically adjusts the generated art data and projects it into the public space.

[0050] 5. Collaboration that combines local elements

[0051] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[0052] Specific examples

[0053] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[0054] Server: Initial setup and art design acquisition

[0055] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[0056] Generative AI: Generating Interactive Art

[0057] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically in response to the movements of passersby.

[0058] Terminal: Receiving art data and preparing for projection

[0059] The generated art is sent to the device, which then prepares it for projection onto the wall. The device monitors the movements of passersby in real time via sensors.

[0060] User (passerby): Motion detection and art changes

[0061] Sensors detect the movements of passersby and send that data to a device. For example, if a passerby approaches the wall and waves, the art changes color and the scenery changes accordingly.

[0062] Collaboration with the local community

[0063] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[0064] This system will provide passersby with a new art experience, while also drawing out the charm of the local area and promoting its revitalization.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] The server retrieves the artist's design from the database. This is a process of searching for design data using the specified artist ID or project ID and downloading the necessary files.

[0068] Step 2:

[0069] The server sends the acquired design data to the generation AI, which includes tags and metadata for adding interactive elements. The generation AI analyzes the design data and generates interactive elements (e.g., color changes, shape changes) that change in response to the movements of passersby.

[0070] Step 3:

[0071] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[0072] Step 4:

[0073] The device receives the generated interactive art data from the server, stores the data in a buffer, and performs initialization to display the art in real time.

[0074] Step 5:

[0075] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, measuring the user's distance and type of movement (e.g., waving, jumping, etc.).

[0076] Step 6:

[0077] The sensor data is sent to the device in real time, and the device analyzes the received sensor data to identify the user's actions.

[0078] Step 7:

[0079] The device dynamically readjusts the generated interactive art based on data from sensors: for example, the art changes color when the user approaches, or rotates when the user waves their hand.

[0080] Step 8:

[0081] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a wall or other public space. The projector displays the art in high resolution.

[0082] Step 9:

[0083] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[0084] Step 10:

[0085] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[0086] Step 11:

[0087] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[0088] Step 12:

[0089] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[0090] Example 1

[0091] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0092] Traditional art exhibitions and installations have often been static, with limited interaction with the audience. Furthermore, there has been a lack of efforts to create dynamic art that incorporates the characteristics and culture of the local area. This has limited the appeal of art to audiences and placed limitations on local promotional activities. As a result, there has been a demand for ways to strengthen the connections between art, audiences, and the local area.

[0093] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0094] In this invention, the server includes a means for acquiring designs, a generating means for analyzing the acquired designs and generating elements that change in response to movement, a terminal means for receiving and preparing the generated interactive art data, a sensor means for detecting movement, a projection means for dynamically adjusting the art data based on data from the sensor and projecting it in a public space, and a linking means for collecting data on local historical elements, characters, etc. and providing it to the generating means. This enables interactive art exhibitions, where art changes in real time in response to the movement of visitors, strengthening ties with the local community. It can also provide a new art experience for local promotional activities.

[0095] A "design" refers to an artist's visual representation or creation, including tags and metadata.

[0096] "Generation means" refers to a means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby or spectators.

[0097] "Terminal means" refers to a device or system that receives the generated interactive art data, stores it, and prepares it for projection.

[0098] "Sensor means" refers to devices or systems that detect the movements of passersby and spectators and collect that data in real time.

[0099] "Projection Measure" refers to a device or system for projecting dynamically adjusted artistic data into a public space based on data from sensors.

[0100] "Collaboration means" refers to devices or systems that collect data such as historical elements and characters of a region, provide it to the generation means, and use it to generate interactive art.

[0101] "Public place" refers to any outdoor or indoor space that is freely accessible to passersby and spectators.

[0102] This invention is a system that acquires designs drawn by artists, analyzes them using a generative AI model, and generates and projects interactive art. The system's main components are a server, a terminal, and a user.

[0103] Overall system picture

[0104] This system is designed to provide interactive art experiences in public spaces. The main hardware used includes a database management system, servers, sensors, high-performance PCs, and projectors. The software used includes generative AI models and real-time processing libraries. Specific examples include MySQL (registered trademark) and PostgreSQL for the database management system, Amazon Web Services (AWS (registered trademark)) and Google (registered trademark) Cloud Platform (GCP) for the servers, OpenAI (registered trademark) GPT-4 (registered trademark) and DALL-E 2 for the generative AI models, and OpenCV for real-time processing.

[0105] Server acquires and analyzes the design

[0106] The server first retrieves the artist's design from a database. This design contains tags and metadata, and the generative AI model analyzes it based on this information. Once the analysis begins, the generative AI extracts each element in the design and adds interactive features, such as elements that change color or shape when passersby approach.

[0107] Generating interactive art using generative AI models

[0108] The generative AI model generates interactive art based on the data received from the server. Specifically, a prompt is provided to the generative AI model, which then generates art with dynamic elements based on that prompt.

[0109] Data reception by terminal and preparation for projection

[0110] The interactive art data generated from the server is sent to the device. The device receives this data and stores it in a buffer. It then prepares for projection and checks and initializes the sensor settings. This ensures that the device is always ready to project the latest art data.

[0111] User movement detection and processing

[0112] When users (passersby) move around in public spaces, sensors detect their movements in real time. The sensors accurately measure the distance and actions of passersby (such as waving or jumping) and send that data to the device. The device then uses this data to dynamically adjust and project the generated art.

[0113] Collaboration with the local community

[0114] The server collects data on historical elements and popular characters of the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines this data with the art to generate new interactive art. This data is then sent back to the device via the server and used in local promotional activities and events.

[0115] Examples of specific examples and prompts

[0116] As a specific scenario, consider the projection of new art in Shibuya, Tokyo. The server acquires Shibuya-themed art designs and has the generation AI analyze them. The generation AI generates interactive art that changes the time of day from night to day when passersby approach. This data is sent to the device and projected by the projector. Sensors detect the movement of passersby, causing the art to change dynamically.

[0117] Prompt Sentence Examples

[0118] "Analyze an art design containing iconic Shibuya scenery and generate interactive art that changes the time of day from night to day when passersby approach."

[0119] This system will provide passersby with a new art experience, draw out the charm of the area, and promote revitalization.

[0120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0121] Program processing steps

[0122] Step 1: Obtain the design and request analysis

[0123] input

[0124] Design data in the database

[0125] output

[0126] Analysis prompts and design information sent to the generative AI

[0127] Specific actions

[0128] The server retrieves the artist's designs from the database, which contain tags and metadata, analyzes the designs, and prepares to send prompts to the generation AI.

[0129] sql

[0130] SELECT FROM designs WHERE artist_id = 'artist123';

[0131] Prompt Sentence Examples

[0132] "Analyze a design containing an iconic view of Shibuya and generate interactive art that changes the time of day from night to day as passersby approach."

[0133] Step 2: Generative AI creates interactive art

[0134] input

[0135] Prompts and design information sent from the server

[0136] output

[0137] Generated interactive art data

[0138] Specific actions

[0139] The generative AI takes design data received from the server and generates interactive art based on specified prompts, such as programming elements in the design to change in response to movement.

[0140] python

[0141] prompt = "A design featuring the iconic Shibuya scenery, with added details that change based on the movement of passersby."

[0142] ai_response = gpt4.generate(prompt)

[0143] Step 3: Receiving and preparing the art data on your device

[0144] input

[0145] Interactive art data generated by generative AI

[0146] output

[0147] Art data stored in the buffer, initial sensor settings

[0148] Specific actions

[0149] The server sends the generated interactive art data to the device using a REST API. The device stores the received data in a buffer, prepares for projection, and checks and initializes the sensor settings.

[0150] python

[0151] response = requests.post('http: / / endpoint_to_terminal', json=art_data)

[0152] with open('art_data_buffer.bin', 'wb') as buffer_file:

[0153] buffer_file.write(response.content)

[0154] Step 4: Detecting user movements and processing data

[0155] input

[0156] User movement data detected by sensors (e.g., distance, type of movement)

[0157] output

[0158] Dynamically adjusted art data

[0159] Specific actions

[0160] As users move through public spaces, sensors detect their movements in real time, measuring the distance and type of movement of passersby and transmitting that data to the device, which then dynamically adjusts and projects the generated art.

[0161] python

[0162] sensor_data = sensor.read_data()

[0163] if sensor_data['distance'] < threshold:

[0164] update_art_display('interaction_type', 'color_change')

[0165] Step 5: Collect and fuse local data

[0166] input

[0167] Regional historical elements and character data

[0168] output

[0169] New prompts and design data provided to the generative AI

[0170] Specific actions

[0171] The server collects data on local historical elements and popular characters, and instructs the AI ​​to analyze it. The AI ​​then combines this data into interactive art to generate new art data, which can be used in local events and promotional activities.

[0172] sql

[0173] SELECT FROM regional_data WHERE region = 'Shibuya';

[0174] Prompt Sentence Examples

[0175] "Create interactive art that includes historical elements and famous characters from Shibuya."

[0176] This will enable interactive art exhibitions, allowing the art to change in real time in response to the movements of visitors, and will also strengthen ties with the local community.

[0177] (Application example 1)

[0178] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0179] In modern public facilities and commercial areas, static decorations and advertisements do not have enough impact on passersby, making it difficult to increase customer engagement. It is also difficult to bring out the unique charm of a region without utilizing its historical elements and character. Furthermore, it is necessary to provide a more engaging experience by incorporating interactive elements that recognize passersby's movements and behavior in real time and dynamically change accordingly. A new system is needed to solve these issues, increase customer engagement, and improve the appeal of commercial areas and public facilities.

[0180] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0181] In this invention, the server includes means for acquiring designs drawn by artists, means for analyzing the acquired designs and generating interactive elements that change in response to the movements of passersby, terminal means for receiving and preparing the generated interactive art data, sensor means for detecting the movements of passersby, projection means for dynamically adjusting the art data based on the data from the sensors and projecting it in a public space, collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation means, implementation means for projecting the generated interactive art as a decorative display inside the store, and means for increasing customer engagement through the interactive art. This makes it possible to provide a more attractive and interactive experience and increase customer engagement compared to static decorations or advertisements.

[0182] An "artist-drawn design" is a work of art created by an artist, expressed as a visual element and including tags and metadata.

[0183] "Means for obtaining" refers to the methods and technical devices for obtaining the artist-drawn designs from the database.

[0184] "Analysis and generation of interactive elements that change in response to the movements of passersby" refers to the method and technical device for analyzing an artist's design using generative AI and generating interactive art that includes dynamic elements that change in response to the movements of passersby.

[0185] "Terminal means for receiving and preparing generated interactive art data" refers to a terminal device that receives interactive art data sent from the server and prepares it for projection.

[0186] "Sensor means for detecting the movement of passersby" refers to various sensors and their systems for detecting the distance and movement of passersby in real time.

[0187] "Projection means for dynamically adjusting art data based on data from sensors and projecting it into a public space" refers to a method and technical device for updating interactive art data based on data acquired by sensors and using a projector to project art into a public space.

[0188] "Collaboration means for collecting data on regional historical elements, characters, etc. and providing it to the generation means" refers to the methods and technical devices for collecting historical aspects and character information related to a specific region and providing it to the generation AI.

[0189] "Interior store decorative displays" refer to devices that are installed on walls, floors, etc. inside commercial facilities to display designs and information.

[0190] "Means to increase customer engagement" refers to methods and technological devices that increase customer relationships and interest through interactive experiences and art.

[0191] This invention is a system that enhances customer engagement in brick-and-mortar stores through interactive art based on artist designs. The system is primarily comprised of a server, terminals, and users, each of which plays a role in generating and projecting interactive art.

[0192] Server Roles

[0193] The server first retrieves the artist's design from the database. This design is visually represented and includes tags and metadata as additional information. The server then passes the retrieved design to a generation AI, which analyzes each element in the design and generates interactive art. The generated interactive art data is then sent from the server to the device.

[0194] Device Role

[0195] The device receives the interactive art data sent from the server and stores it in a buffer. The device prepares for projection and also checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest interactive art.

[0196] User Roles

[0197] As users (customers) move around the store, sensors detect their movements. The sensors accurately measure the user's distance and actions (e.g., waving, jumping, etc.). The device that receives this data dynamically adjusts the generated art data and projects it onto the walls and floors inside the store.

[0198] Hardware and Software Details

[0199] Hardware

[0200] Cameras and sensors: to detect user movement and distance

[0201] Projector: for projecting interactive art

[0202] software

[0203] OpenCV: for image processing and display

[0204] TENSORFLOW® / Keras: for using generative AI models

[0205] The server retrieves the artist's design and uses generative AI to analyze and generate interactive art. The generated art data is adjusted in real time based on customer movements detected by sensors and projected inside the store via a projector. This provides a more engaging and interactive experience than static decorations or advertisements, and can improve customer engagement.

[0206] Specific examples and prompts for the generative AI model

[0207] Specific examples

[0208] Art projected onto the walls of the shopping mall changes into a flower design when passersby approach, and it also recognizes certain actions (e.g. jumping) and displays special effects according to the action.

[0209] Example prompts

[0210] Artist Design: Cityscape

[0211] Effect: The city changes from day to night as passersby approach

[0212] Interaction: Buildings change color when passersby wave

[0213] Regional elements: Designs incorporating regional history and famous characters

[0214] This example shows how store décor can go beyond being just a visual element to provide customers with a dynamic and interactive experience, capturing their attention, increasing their dwell time in the store, and increasing their engagement.

[0215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0216] Step 1:

[0217] The server retrieves the artist's drawing from the database.

[0218] Input: Artist's design ID

[0219] Data processing: Search the database based on the design ID and obtain the corresponding design data

[0220] Output: Artist's design (image file and additional information)

[0221] Step 2:

[0222] The server passes the acquired design to the generative AI model and begins analysis.

[0223] Input: Artist's design (image file and additional information)

[0224] Data Computation: Generative AI models analyze the visual elements of a design and generate dynamic, interactive elements.

[0225] Output: Interactive art data

[0226] Step 3:

[0227] The server transmits the generated interactive art data to the terminal.

[0228] Input: Interactive art data

[0229] Data processing: Converting interactive art data into a format that can be received by the device

[0230] Output: Converted interactive art data

[0231] Step 4:

[0232] The terminal stores the received interactive art data in a buffer and prepares it for projection.

[0233] Input: Generated interactive art data

[0234] Data processing: Save interactive art data to a buffer, check and initialize projector settings

[0235] Output: The projector is ready to project.

[0236] Step 5:

[0237] As users (customers) move around the store, sensors detect their movements.

[0238] Input: User movement (distance and action)

[0239] Data processing: Motion detected by sensors is converted into data in real time

[0240] Output: Detected motion data

[0241] Step 6:

[0242] The device dynamically adjusts the art data based on data from the sensors and performs the projection.

[0243] Input: User movement data from sensors, interactive art data

[0244] Data Calculation: Dynamically update the color and shape of interactive art based on user movement

[0245] Output: Updated interactive art (projection)

[0246] Step 7:

[0247] The server collects data on the region's historical elements, characters, etc. and provides it to the generative AI model.

[0248] Input: Regional historical data, character data

[0249] Data processing: Converting collected data into a format that can be analyzed by the AI ​​model

[0250] Output: Interactive art data including regional data

[0251] Step 8:

[0252] The terminal then projects the final interactive art as a decorative display inside the store.

[0253] Input: Final generated interactive art data

[0254] Data processing: Projecting interactive art data optimized for display

[0255] Output: Interactive art projected onto decorative displays inside the store

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[0258] System Overview

[0259] 1. The server acquires and analyzes the artist's design

[0260] The first step the server takes is to retrieve the artist's design from the database. The retrieved design includes the artist's intended visual expression as well as tags and metadata for adding interactive elements. The server then sends the retrieved design data to the generation AI, instructing it to generate interactive elements that change in response to the movements and emotions of passersby.

[0261] 2. Generating Interactive Art with Generative AI

[0262] The generative AI analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change according to the movements and emotions of passersby. This data is then sent to the device via the server.

[0263] 3. Data reception by the terminal and preparation for projection

[0264] The device receives the generated interactive art data from the server, stores this data in a buffer, and initializes the device to display the art in real time. Additionally, the device prepares to receive data from the sensors and emotion engine in real time.

[0265] 4. User movement detection and emotion recognition

[0266] When a user (passerby) passes through a public space, a sensor detects their movement. The data measured by the sensor is sent to the device in real time. At the same time, the emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[0267] 5. Device processing of sensor and emotion data

[0268] The device receives and analyzes movement data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, the color and shape of the art may change when the user approaches, or the color may become brighter if the user smiles.

[0269] 6. Projecting art using projection media

[0270] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[0271] 7. Collaboration that combines local elements

[0272] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[0273] Specific examples

[0274] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[0275] Server: Initial setup and art design acquisition

[0276] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[0277] Generative AI: Generating Interactive Art

[0278] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically according to the movements and emotions of passersby.

[0279] Terminal: Receiving art data and preparing for projection

[0280] The generated art is sent to the device, which prepares it for projection onto the wall. The device monitors passersby's movements via sensors and their emotions via an emotion engine in real time.

[0281] User (passerby): Motion and emotion detection, and art changes

[0282] Sensors detect the movements of passersby, and an emotion engine analyzes their facial expressions to identify their emotions. This data is then sent to the device. For example, if a passerby approaches the wall and smiles, the colors of the artwork will become more vibrant and the scenery will change to reflect their emotions.

[0283] Collaboration with the local community

[0284] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[0285] This system provides passersby with a deeper art experience, drawing out the local area's charm and promoting its revitalization. In addition, by combining it with an emotion engine, it becomes possible to provide individual experiences that correspond to the user's emotions.

[0286] The processing flow will be explained below.

[0287] Step 1:

[0288] The server retrieves the artist's design from the database. Specifically, it searches for the design data using the specified artist ID or project ID and downloads the necessary files.

[0289] Step 2:

[0290] The server sends the acquired design data, which includes tags and metadata for adding interactive elements, to the generation AI, which analyzes the design data and generates interactive elements that change in response to the movements and emotions of passersby.

[0291] Step 3:

[0292] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[0293] Step 4:

[0294] The device receives the generated interactive art data from the server. The device stores this data in a buffer and performs initial setup to display the art in real time. It also prepares settings to receive data from the emotion engine and sensors.

[0295] Step 5:

[0296] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, by measuring the user's distance and type of action (e.g., waving, jumping, etc.).

[0297] Step 6:

[0298] The emotion engine analyzes the user's facial expressions and recognizes their emotions. Specifically, it uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[0299] Step 7:

[0300] The sensor and emotion engine send measured data to the device in real time, and the device analyzes the received sensor data and emotion data to identify the user's actions and emotions.

[0301] Step 8:

[0302] The device dynamically readjusts the generated interactive art based on data from sensors and an emotion engine, for example, changing the color of the art when the user approaches, or turning brighter if the user smiles.

[0303] Step 9:

[0304] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[0305] Step 10:

[0306] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[0307] Step 11:

[0308] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[0309] Step 12:

[0310] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[0311] Step 13:

[0312] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[0313] Example 2

[0314] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] In systems that project interactive art in public spaces, there is a need to provide a richer art experience by reflecting not only the movements but also the emotions of passersby. Another important issue is to contribute to the revitalization of the local area by incorporating local history and culture.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation AI means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means for detecting the movements of passersby; emotion recognition means including an emotion engine that recognizes the emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion recognition means and projecting it in a public place; and collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation AI means. This makes it possible to generate interactive art that changes color and shape based on the movements and emotions of passersby and is used in local events and promotional activities.

[0317] "Artist-drawn design" means design data created by an artist that includes visual expressions and interactive elements.

[0318] "Generative AI means" is an artificial intelligence technology that analyzes the artist's design and generates interactive elements that change in response to the movements and emotions of passersby.

[0319] "Terminal means" refers to a device that receives interactive art data generated from the server and prepares it for real-time display and projection.

[0320] "Sensor means" refers to equipment or technology used to detect the movement of passersby.

[0321] The "emotion recognition means" is an emotion engine that includes face recognition technology to recognize the emotions of passersby.

[0322] "Projection means" means a device for projecting adjusted art data into a public space.

[0323] "Collaboration means" refers to mechanisms or means for collecting data on local historical elements, characters, etc., and providing it to the generative AI means.

[0324] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[0325] The server retrieves the artist's design from the database. The retrieved design includes visual and interactive elements. The server sends this design data to the generation AI, instructing it to generate interactive elements that change according to the movements and emotions of passersby. For example, the server retrieves the design data from a MySQL database and sends a POST request to the generation AI's API.

[0326] The generative AI then analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change in response to the movements and emotions of passersby. For example, it includes instructions such as "change color when a user approaches." This data is sent to the device via the server.

[0327] The device receives the generated interactive art data from the server and stores it in a buffer. The device prepares to receive data from sensors and emotion engines in real time. For example, the device receives art data from an AWS EC2 server and stores it in local storage.

[0328] When a user (passerby) passes through a public space, a sensor detects their movement. The data captured by the sensor is sent to the device in real time. At the same time, an emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger. For example, the sensor uses Microsoft® Kinect, and the emotion engine uses Affectiva SDK.

[0329] The device receives and analyzes motion data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. This adjusted art data is then sent to the projection device.

[0330] The projection device projects the adjusted interactive art data in high resolution into a public space. The projector continues to display the art while waiting for the next sensor and emotion data. For example, an EPSON projector is used to project the art.

[0331] Another feature of this system is its collaboration with local elements. The server collects data on the region's history, culture, popular characters, and other topics, and sends it to the AI ​​generator. The AI ​​analyzes this data and generates new interactive art. This art data is then sent back to the device via the server and used in local promotional events.

[0332] An example of a prompt is, "Using design data themed on the city of Shibuya, generate interactive art in which the scenery changes from night to day when passersby approach, and the colors change vividly when passersby smile." By inputting this prompt into a generative AI model, specific art is generated.

[0333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0334] Step 1:

[0335] The server retrieves the artist's design. Specifically, the server connects to a database (e.g., MySQL) and executes the query "SELECT FROM designs WHERE artist_id = '12345'". The input is the database connection information and the query, and the output is the retrieved design data. This design data includes visual and interactive elements.

[0336] Step 2:

[0337] The server sends the acquired design data to the generative AI. Specifically, the server sends an HTTP POST request to the API endpoint of the generative AI model, including the query result design data in JSON format. The input is the acquired design data, and the output is interactive art data generated by the generative AI model.

[0338] Step 3:

[0339] The generative AI analyzes the design data received from the server and generates interactive art. In this process, the generative AI analyzes the visual and interactive elements contained in the design data and generates art that changes in response to movement and emotions. The input is the design data sent from the server, and the output is the generated interactive art data.

[0340] Step 4:

[0341] The device receives the generated interactive art data from the server and stores it in a buffer. Specifically, the device retrieves the data from the server via an HTTP GET request and stores it in local storage (e.g., / tmp / art_data.json). The input is the interactive art data returned by the generation AI, and the output is the stored art data.

[0342] Step 5:

[0343] When a user (passerby) passes through a public space, a sensor detects their movement. Specifically, the sensor (e.g., Microsoft Kinect) captures the user's movement and transmits the data to a device in real time. The input is the user's movement, and the output is the detected movement data.

[0344] Step 6:

[0345] The emotion engine analyzes the user's facial expressions and recognizes emotions. Specifically, the emotion engine (e.g., Affectiva SDK) captures the user's face in real time and generates emotion data (happiness, surprise, sadness, anger, etc.). The input is the user's facial expression, and the output is the recognized emotion data.

[0346] Step 7:

[0347] The device receives and analyzes sensor movement data and emotion data from the emotion engine. Specifically, the device analyzes the data received in real time and dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. The input is data from the sensor and emotion engine, and the output is the adjusted art data.

[0348] Step 8:

[0349] The terminal transmits the adjusted art data to a projection device, which projects it into a public space. Specifically, the terminal transmits the art data to a projector (e.g., an EPSON projector) and projects it in high resolution. The input is the adjusted interactive art data, and the output is the art projected into a public space.

[0350] Step 9:

[0351] The server collects regional data (history, characters, etc.) and sends it to the generation AI. Specifically, the server collects regional data from APIs and databases and sends it to the generation AI. The input is regional data, and the output is the generated interactive art specific to the region.

[0352] Step 10:

[0353] Interactive art specific to the region is generated and sent to the device. Specifically, the generation AI analyzes the region data received from the server, generates new interactive art, and sends it to the device. The input is the region data from the server, and the output is the generated interactive art data specific to the region.

[0354] (Application example 2)

[0355] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] Currently, public spaces and commercial facilities mainly present static advertisements and art exhibits to passersby and customers, lacking in interactive experiences. Furthermore, there has been little progress in practical application of technology that provides dynamic content that reflects the emotions of passersby and customers. This means that traditional advertisements and exhibits are one-way, making it difficult to attract users' attention. Furthermore, there is a lack of content that reflects local culture and history, making it difficult to contribute to regional revitalization.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means and emotion engine means for detecting the movements and emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion engine and projecting it in a public place or commercial facility; and collaboration means for collecting local cultural elements and providing them to the generation means. This makes it possible to provide passersby and customers with an interactive, dynamic art experience that responds to their emotions, and to contribute to regional revitalization with content that reflects the local culture and history.

[0358] "Means for capturing artist-drawn designs" refers to any equipment or software used to receive or capture in digital form the visual designs created by the artist.

[0359] "Generation means" refers to software or algorithms that generate interactive elements based on acquired design data.

[0360] "Terminal means" refers to a process or device for receiving, displaying, and projecting the generated art data.

[0361] "Sensor means" refers to devices or technologies for detecting the movement of users or passersby.

[0362] "Emotion engine means" refers to software or algorithms that analyze the facial expressions of users and passersby and identify their emotions.

[0363] "Projection means" refers to a projector or related device for projecting art data onto a wall or other surface in a public place or commercial facility.

[0364] "Collaboration tools" refer to databases and software systems that collect local cultural elements and provide them to production tools.

[0365] MODE FOR CARRYING OUT THE INVENTION

[0366] Overall system configuration

[0367] This invention is a system for projecting interactive art in public places and commercial facilities based on designs drawn by artists. This system is composed of a generation means for analyzing and generating acquired design data, a sensor means and emotion engine means for detecting movement and emotions, a terminal means for receiving and displaying data, a projection means for projecting the art, and a collaboration means for collecting local cultural elements.

[0368] Specific examples of each component

[0369] server

[0370] The server retrieves and analyzes the artist's design. The design data includes tags and metadata for adding interactive elements. The server then sends this design data to the AI, instructing it to generate interactive art based on the movements and emotions of passersby.

[0371] generation means

[0372] The generation method uses a generative AI model (e.g., OpenAI's DALL-E or GPT-4), analyzes the design data received from the server, and generates interactive art that changes in response to the user's movements and emotions.

[0373] Terminal means

[0374] The terminal means receives the generated interactive art data, stores the data in a buffer, performs initialization for displaying the art in real time, and prepares to receive data from the sensor means and emotion engine means in real time.

[0375] Sensor Means and Emotion Engine Means

[0376] The sensor means uses cameras and IR sensors, which detect the movements of users (passersby) in real time. The emotion engine means uses Microsoft Azure (registered trademark) Cognitive Services API and other tools to analyze facial expressions and identify emotions such as joy, surprise, sadness, and anger.

[0377] Projection Method

[0378] A high-resolution projector is used as a projection medium, and the generated interactive art data is projected into public spaces or commercial facilities.

[0379] Collaboration tools

[0380] The server collects local cultural elements and provides them to the generation means, allowing the generation AI to generate interactive art that incorporates new elements.

[0381] Example

[0382] Specific examples include the following:

[0383] Interactive art in public spaces

[0384] The server retrieves designs that reflect the characteristics of the area, and the AI ​​generates interactive art that changes in response to the movements and emotions of passersby.

[0385] Example prompt sentence:

[0386] Generate an art piece that changes color and form based on user emotions and movements detected via sensors.

[0387] The device receives data from the sensors and configures it to display art in real time.

[0388] When passersby smile, the projected art changes into vibrant colors and adds animation effects.

[0389] Art exhibition in a commercial facility

[0390] The server retrieves designs based on the theme of the commercial facility and generates interactive art using generative AI.

[0391] The device displays art on its display and receives data from sensors and an emotion engine.

[0392] As customers pass by the display in the store, the art changes based on their movements and facial expressions, providing a more enjoyable shopping experience.

[0393] In this way, the system implementing the invention can effectively combine an improved art experience with local cultural elements.

[0394] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0395] Step 1:

[0396] The server retrieves the artist's design from the database. When retrieving the design data, it also includes tags and metadata for adding interactive elements. This becomes the input data for the server. The retrieved design data is then analyzed in digital format and processed into a format that can be sent to the generation AI. This processed data becomes the output data from the server to the generation AI.

[0397] Step 2:

[0398] The server sends the acquired design data to the generation AI, which then generates interactive art based on a prompt (e.g., "Generate an art piece that changes color and form based on user emotions and movements detected via sensors."). The generation AI analyzes the data and generates art, and the generated interactive art data is the output.

[0399] Step 3:

[0400] The server receives the interactive art data generated by the generation AI and sends it to the terminal. The terminal stores the received data in a buffer and performs initial settings to display the art in real time. This becomes input data to the terminal. The terminal prepares to receive data from the sensor means and emotion engine means in real time.

[0401] Step 4:

[0402] When a user (passerby) visits a public place or commercial facility, the sensor means detects the user's movements, and at the same time the emotion engine means analyzes the user's facial expressions. The movement data and emotion data acquired by the sensor and emotion engine become input data to the terminal. Based on this data, the terminal performs data analysis and dynamically adjusts the art data.

[0403] Step 5:

[0404] The device adjusts the interactive art data in real time based on the motion and emotion data. For example, it can change the color or shape of the art when the user approaches, or change to a brighter color tone when the user smiles. This adjusted art data becomes input data for the projection means.

[0405] Step 6:

[0406] The device then transmits the adjusted interactive art data to a projector, which projects it into a public or commercial space. The projector displays the art in high resolution and waits for new sensor and emotion data input. The displayed art allows users to enjoy an interactive experience.

[0407] Step 7:

[0408] The server collects local cultural and historical elements from a database and provides this data to a generation AI, which then generates new interactive art incorporating elements unique to the region or commercial facility. This newly generated art data becomes output data for the server and can be used again on devices or projection means.

[0409] This process brings together the server, the device, and the user to create a dynamic interactive art experience.

[0410] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0411] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0412] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0413] [Second embodiment]

[0414] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0415] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0416] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0417] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0418] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0419] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0420] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0421] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0422] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0423] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0424] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0425] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0426] This invention is a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists. The system's main components are a server, a terminal, and a user.

[0427] System Overview

[0428] 1. The server acquires and analyzes the artist's design

[0429] The first step the server takes is to retrieve the artist's design from the database. This design is visually represented and includes tags and metadata. The server then asks the generation AI to analyze the retrieved design. Once the generation AI begins its analysis, it extracts each element in the design and adds interactive elements (for example, parts that change color depending on how close passersby are).

[0430] 2. Generating Interactive Art with Generative AI

[0431] The AI ​​analyzes the design received from the server and begins generating interactive art. The generated art has dynamic elements, such as changing color or shape when passersby approach. The generated interactive art data is then returned to the server.

[0432] 3. Data reception by the terminal and preparation for projection

[0433] The server sends the generated interactive art data to the device. The device stores the received art data in a buffer and prepares for projection. At this time, the device checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest art data.

[0434] 4. Detecting and Processing User Movements

[0435] When a user (passerby) moves through a public space, sensors detect their movements. The data measured by the sensors is sent to the device in real time. The sensors accurately measure the passerby's distance and type of movement (e.g., waving, jumping, etc.). The device receives this data and dynamically adjusts the generated art data and projects it into the public space.

[0436] 5. Collaboration that combines local elements

[0437] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[0438] Specific examples

[0439] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[0440] Server: Initial setup and art design acquisition

[0441] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[0442] Generative AI: Generating Interactive Art

[0443] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically in response to the movements of passersby.

[0444] Terminal: Receiving art data and preparing for projection

[0445] The generated art is sent to the device, which then prepares it for projection onto the wall. The device monitors the movements of passersby in real time via sensors.

[0446] User (passerby): Motion detection and art changes

[0447] Sensors detect the movements of passersby and send that data to a device. For example, if a passerby approaches the wall and waves, the art changes color and the scenery changes accordingly.

[0448] Collaboration with the local community

[0449] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[0450] This system will provide passersby with a new art experience, while also drawing out the charm of the local area and promoting its revitalization.

[0451] The processing flow will be explained below.

[0452] Step 1:

[0453] The server retrieves the artist's design from the database. This is a process of searching for design data using the specified artist ID or project ID and downloading the necessary files.

[0454] Step 2:

[0455] The server sends the acquired design data to the generation AI, which includes tags and metadata for adding interactive elements. The generation AI analyzes the design data and generates interactive elements (e.g., color changes, shape changes) that change in response to the movements of passersby.

[0456] Step 3:

[0457] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[0458] Step 4:

[0459] The device receives the generated interactive art data from the server, stores the data in a buffer, and performs initialization to display the art in real time.

[0460] Step 5:

[0461] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, measuring the user's distance and type of movement (e.g., waving, jumping, etc.).

[0462] Step 6:

[0463] The sensor data is sent to the device in real time, and the device analyzes the received sensor data to identify the user's actions.

[0464] Step 7:

[0465] The device dynamically readjusts the generated interactive art based on data from sensors: for example, the art changes color when the user approaches, or rotates when the user waves their hand.

[0466] Step 8:

[0467] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a wall or other public space. The projector displays the art in high resolution.

[0468] Step 9:

[0469] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[0470] Step 10:

[0471] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[0472] Step 11:

[0473] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[0474] Step 12:

[0475] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[0476] Example 1

[0477] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0478] Traditional art exhibitions and installations have often been static, with limited interaction with the audience. Furthermore, there has been a lack of efforts to create dynamic art that incorporates the characteristics and culture of the local area. This has limited the appeal of art to audiences and placed limitations on local promotional activities. As a result, there has been a demand for ways to strengthen the connections between art, audiences, and the local area.

[0479] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0480] In this invention, the server includes a means for acquiring designs, a generating means for analyzing the acquired designs and generating elements that change in response to movement, a terminal means for receiving and preparing the generated interactive art data, a sensor means for detecting movement, a projection means for dynamically adjusting the art data based on data from the sensor and projecting it in a public space, and a linking means for collecting data on local historical elements, characters, etc. and providing it to the generating means. This enables interactive art exhibitions, where art changes in real time in response to the movement of visitors, strengthening ties with the local community. It can also provide a new art experience for local promotional activities.

[0481] A "design" refers to an artist's visual representation or creation, including tags and metadata.

[0482] "Generation means" refers to a means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby or spectators.

[0483] "Terminal means" refers to a device or system that receives the generated interactive art data, stores it, and prepares it for projection.

[0484] "Sensor means" refers to devices or systems that detect the movements of passersby and spectators and collect that data in real time.

[0485] "Projection Measure" refers to a device or system for projecting dynamically adjusted artistic data into a public space based on data from sensors.

[0486] "Collaboration means" refers to devices or systems that collect data such as historical elements and characters of a region, provide it to the generation means, and use it to generate interactive art.

[0487] "Public place" refers to any outdoor or indoor space that is freely accessible to passersby and spectators.

[0488] This invention is a system that acquires designs drawn by artists, analyzes them using a generative AI model, and generates and projects interactive art. The system's main components are a server, a terminal, and a user.

[0489] Overall system picture

[0490] This system is designed to provide interactive art experiences in public spaces. The main hardware used includes a database management system, servers, sensors, high-performance PCs, and projectors. The software used includes generative AI models and libraries for real-time processing. Specifically, the database management system uses MySQL or PostgreSQL, the servers use Amazon Web Services (AWS) or Google Cloud Platform (GCP), the generative AI models use OpenAI GPT-4 or DALL-E 2, and the real-time processing uses OpenCV.

[0491] Server acquires and analyzes the design

[0492] The server first retrieves the artist's design from a database. This design contains tags and metadata, and the generative AI model analyzes it based on this information. Once the analysis begins, the generative AI extracts each element in the design and adds interactive features, such as elements that change color or shape when passersby approach.

[0493] Generating interactive art using generative AI models

[0494] The generative AI model generates interactive art based on the data received from the server. Specifically, a prompt is provided to the generative AI model, which then generates art with dynamic elements based on that prompt.

[0495] Data reception by terminal and preparation for projection

[0496] The interactive art data generated from the server is sent to the device. The device receives this data and stores it in a buffer. It then prepares for projection and checks and initializes the sensor settings. This ensures that the device is always ready to project the latest art data.

[0497] User movement detection and processing

[0498] When users (passersby) move around in public spaces, sensors detect their movements in real time. The sensors accurately measure the distance and actions of passersby (such as waving or jumping) and send that data to the device. The device then uses this data to dynamically adjust and project the generated art.

[0499] Collaboration with the local community

[0500] The server collects data on historical elements and popular characters of the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines this data with the art to generate new interactive art. This data is then sent back to the device via the server and used in local promotional activities and events.

[0501] Examples of specific examples and prompts

[0502] As a specific scenario, consider the projection of new art in Shibuya, Tokyo. The server acquires Shibuya-themed art designs and has the generation AI analyze them. The generation AI generates interactive art that changes the time of day from night to day when passersby approach. This data is sent to the device and projected by the projector. Sensors detect the movement of passersby, causing the art to change dynamically.

[0503] Prompt Sentence Examples

[0504] "Analyze an art design containing iconic Shibuya scenery and generate interactive art that changes the time of day from night to day when passersby approach."

[0505] This system will provide passersby with a new art experience, draw out the charm of the area, and promote revitalization.

[0506] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0507] Program processing steps

[0508] Step 1: Obtain the design and request analysis

[0509] input

[0510] Design data in the database

[0511] output

[0512] Analysis prompts and design information sent to the generative AI

[0513] Specific actions

[0514] The server retrieves the artist's designs from the database, which contain tags and metadata, analyzes the designs, and prepares to send prompts to the generation AI.

[0515] sql

[0516] SELECT FROM designs WHERE artist_id = 'artist123';

[0517] Prompt Sentence Examples

[0518] "Analyze a design containing an iconic view of Shibuya and generate interactive art that changes the time of day from night to day as passersby approach."

[0519] Step 2: Generative AI creates interactive art

[0520] input

[0521] Prompts and design information sent from the server

[0522] output

[0523] Generated interactive art data

[0524] Specific actions

[0525] The generative AI takes design data received from the server and generates interactive art based on specified prompts, such as programming elements in the design to change in response to movement.

[0526] python

[0527] prompt = "A design featuring the iconic Shibuya scenery, with added details that change based on the movement of passersby."

[0528] ai_response = gpt4.generate(prompt)

[0529] Step 3: Receiving and preparing the art data on your device

[0530] input

[0531] Interactive art data generated by generative AI

[0532] output

[0533] Art data stored in the buffer, initial sensor settings

[0534] Specific actions

[0535] The server sends the generated interactive art data to the device using a REST API. The device stores the received data in a buffer, prepares for projection, and checks and initializes the sensor settings.

[0536] python

[0537] response = requests.post('http: / / endpoint_to_terminal', json=art_data)

[0538] with open('art_data_buffer.bin', 'wb') as buffer_file:

[0539] buffer_file.write(response.content)

[0540] Step 4: Detecting user movements and processing data

[0541] input

[0542] User movement data detected by sensors (e.g., distance, type of movement)

[0543] output

[0544] Dynamically adjusted art data

[0545] Specific actions

[0546] As users move through public spaces, sensors detect their movements in real time, measuring the distance and type of movement of passersby and transmitting that data to the device, which then dynamically adjusts and projects the generated art.

[0547] python

[0548] sensor_data = sensor.read_data()

[0549] if sensor_data['distance'] < threshold:

[0550] update_art_display('interaction_type', 'color_change')

[0551] Step 5: Collect and fuse local data

[0552] input

[0553] Regional historical elements and character data

[0554] output

[0555] New prompts and design data provided to the generative AI

[0556] Specific actions

[0557] The server collects data on local historical elements and popular characters, and instructs the AI ​​to analyze it. The AI ​​then combines this data into interactive art to generate new art data, which can be used in local events and promotional activities.

[0558] sql

[0559] SELECT FROM regional_data WHERE region = 'Shibuya';

[0560] Prompt Sentence Examples

[0561] "Create interactive art that includes historical elements and famous characters from Shibuya."

[0562] This will enable interactive art exhibitions, allowing the art to change in real time in response to the movements of visitors, and will also strengthen ties with the local community.

[0563] (Application example 1)

[0564] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0565] In modern public facilities and commercial areas, static decorations and advertisements do not have enough impact on passersby, making it difficult to increase customer engagement. It is also difficult to bring out the unique charm of a region without utilizing its historical elements and character. Furthermore, it is necessary to provide a more engaging experience by incorporating interactive elements that recognize passersby's movements and behavior in real time and dynamically change accordingly. A new system is needed to solve these issues, increase customer engagement, and improve the appeal of commercial areas and public facilities.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0567] In this invention, the server includes means for acquiring designs drawn by artists, means for analyzing the acquired designs and generating interactive elements that change in response to the movements of passersby, terminal means for receiving and preparing the generated interactive art data, sensor means for detecting the movements of passersby, projection means for dynamically adjusting the art data based on the data from the sensors and projecting it in a public space, collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation means, implementation means for projecting the generated interactive art as a decorative display inside the store, and means for increasing customer engagement through the interactive art. This makes it possible to provide a more attractive and interactive experience and increase customer engagement compared to static decorations or advertisements.

[0568] An "artist-drawn design" is a work of art created by an artist, expressed as a visual element and including tags and metadata.

[0569] "Means for obtaining" refers to the methods and technical devices for obtaining the artist-drawn designs from the database.

[0570] "Analysis and generation of interactive elements that change in response to the movements of passersby" refers to the method and technical device for analyzing an artist's design using generative AI and generating interactive art that includes dynamic elements that change in response to the movements of passersby.

[0571] "Terminal means for receiving and preparing generated interactive art data" refers to a terminal device that receives interactive art data sent from the server and prepares it for projection.

[0572] "Sensor means for detecting the movement of passersby" refers to various sensors and their systems for detecting the distance and movement of passersby in real time.

[0573] "Projection means for dynamically adjusting art data based on data from sensors and projecting it into a public space" refers to a method and technical device for updating interactive art data based on data acquired by sensors and using a projector to project art into a public space.

[0574] "Collaboration means for collecting data on regional historical elements, characters, etc. and providing it to the generation means" refers to the methods and technical devices for collecting historical aspects and character information related to a specific region and providing it to the generation AI.

[0575] "Interior store decorative displays" refer to devices that are installed on walls, floors, etc. inside commercial facilities to display designs and information.

[0576] "Means to increase customer engagement" refers to methods and technological devices that increase customer relationships and interest through interactive experiences and art.

[0577] This invention is a system that enhances customer engagement in brick-and-mortar stores through interactive art based on artist designs. The system is primarily comprised of a server, terminals, and users, each of which plays a role in generating and projecting interactive art.

[0578] Server Roles

[0579] The server first retrieves the artist's design from the database. This design is visually represented and includes tags and metadata as additional information. The server then passes the retrieved design to a generation AI, which analyzes each element in the design and generates interactive art. The generated interactive art data is then sent from the server to the device.

[0580] Device Role

[0581] The device receives the interactive art data sent from the server and stores it in a buffer. The device prepares for projection and also checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest interactive art.

[0582] User Roles

[0583] As users (customers) move around the store, sensors detect their movements. The sensors accurately measure the user's distance and actions (e.g., waving, jumping, etc.). The device that receives this data dynamically adjusts the generated art data and projects it onto the walls and floors inside the store.

[0584] Hardware and Software Details

[0585] Hardware

[0586] Cameras and sensors: to detect user movement and distance

[0587] Projector: for projecting interactive art

[0588] software

[0589] OpenCV: for image processing and display

[0590] TensorFlow / Keras: for using generative AI models

[0591] The server retrieves the artist's design and uses generative AI to analyze and generate interactive art. The generated art data is adjusted in real time based on customer movements detected by sensors and projected inside the store via a projector. This provides a more engaging and interactive experience than static decorations or advertisements, and can improve customer engagement.

[0592] Specific examples and prompts for the generative AI model

[0593] Specific examples

[0594] Art projected onto the walls of the shopping mall changes into a flower design when passersby approach, and it also recognizes certain actions (e.g. jumping) and displays special effects according to the action.

[0595] Example prompts

[0596] Artist Design: Cityscape

[0597] Effect: The city changes from day to night as passersby approach

[0598] Interaction: Buildings change color when passersby wave

[0599] Regional elements: Designs incorporating regional history and famous characters

[0600] This example shows how store décor can go beyond being just a visual element to provide customers with a dynamic and interactive experience, capturing their attention, increasing their dwell time in the store, and increasing their engagement.

[0601] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0602] Step 1:

[0603] The server retrieves the artist's drawing from the database.

[0604] Input: Artist's design ID

[0605] Data processing: Search the database based on the design ID and obtain the corresponding design data

[0606] Output: Artist's design (image file and additional information)

[0607] Step 2:

[0608] The server passes the acquired design to the generative AI model and begins analysis.

[0609] Input: Artist's design (image file and additional information)

[0610] Data Computation: Generative AI models analyze the visual elements of a design and generate dynamic, interactive elements.

[0611] Output: Interactive art data

[0612] Step 3:

[0613] The server transmits the generated interactive art data to the terminal.

[0614] Input: Interactive art data

[0615] Data processing: Converting interactive art data into a format that can be received by the device

[0616] Output: Converted interactive art data

[0617] Step 4:

[0618] The terminal stores the received interactive art data in a buffer and prepares it for projection.

[0619] Input: Generated interactive art data

[0620] Data processing: Save interactive art data to a buffer, check and initialize projector settings

[0621] Output: The projector is ready to project.

[0622] Step 5:

[0623] As users (customers) move around the store, sensors detect their movements.

[0624] Input: User movement (distance and action)

[0625] Data processing: Motion detected by sensors is converted into data in real time

[0626] Output: Detected motion data

[0627] Step 6:

[0628] The device dynamically adjusts the art data based on data from the sensors and performs the projection.

[0629] Input: User movement data from sensors, interactive art data

[0630] Data Calculation: Dynamically update the color and shape of interactive art based on user movement

[0631] Output: Updated interactive art (projection)

[0632] Step 7:

[0633] The server collects data on the region's historical elements, characters, etc. and provides it to the generative AI model.

[0634] Input: Regional historical data, character data

[0635] Data processing: Converting collected data into a format that can be analyzed by the AI ​​model

[0636] Output: Interactive art data including regional data

[0637] Step 8:

[0638] The terminal then projects the final interactive art as a decorative display inside the store.

[0639] Input: Final generated interactive art data

[0640] Data processing: Projecting interactive art data optimized for display

[0641] Output: Interactive art projected onto decorative displays inside the store

[0642] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0643] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[0644] System Overview

[0645] 1. The server acquires and analyzes the artist's design

[0646] The first step the server takes is to retrieve the artist's design from the database. The retrieved design includes the artist's intended visual expression as well as tags and metadata for adding interactive elements. The server then sends the retrieved design data to the generation AI, instructing it to generate interactive elements that change in response to the movements and emotions of passersby.

[0647] 2. Generating Interactive Art with Generative AI

[0648] The generative AI analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change according to the movements and emotions of passersby. This data is then sent to the device via the server.

[0649] 3. Data reception by the terminal and preparation for projection

[0650] The device receives the generated interactive art data from the server, stores this data in a buffer, and initializes the device to display the art in real time. Additionally, the device prepares to receive data from the sensors and emotion engine in real time.

[0651] 4. User movement detection and emotion recognition

[0652] When a user (passerby) passes through a public space, a sensor detects their movement. The data measured by the sensor is sent to the device in real time. At the same time, the emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[0653] 5. Device processing of sensor and emotion data

[0654] The device receives and analyzes movement data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, the color and shape of the art may change when the user approaches, or the color may become brighter if the user smiles.

[0655] 6. Projecting art using projection media

[0656] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[0657] 7. Collaboration that combines local elements

[0658] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[0659] Specific examples

[0660] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[0661] Server: Initial setup and art design acquisition

[0662] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[0663] Generative AI: Generating Interactive Art

[0664] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically according to the movements and emotions of passersby.

[0665] Terminal: Receiving art data and preparing for projection

[0666] The generated art is sent to the device, which prepares it for projection onto the wall. The device monitors passersby's movements via sensors and their emotions via an emotion engine in real time.

[0667] User (passerby): Motion and emotion detection, and art changes

[0668] Sensors detect the movements of passersby, and an emotion engine analyzes their facial expressions to identify their emotions. This data is then sent to the device. For example, if a passerby approaches the wall and smiles, the colors of the artwork will become more vibrant and the scenery will change to reflect their emotions.

[0669] Collaboration with the local community

[0670] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[0671] This system provides passersby with a deeper art experience, drawing out the local area's charm and promoting its revitalization. In addition, by combining it with an emotion engine, it becomes possible to provide individual experiences that correspond to the user's emotions.

[0672] The processing flow will be explained below.

[0673] Step 1:

[0674] The server retrieves the artist's design from the database. Specifically, it searches for the design data using the specified artist ID or project ID and downloads the necessary files.

[0675] Step 2:

[0676] The server sends the acquired design data, which includes tags and metadata for adding interactive elements, to the generation AI, which analyzes the design data and generates interactive elements that change in response to the movements and emotions of passersby.

[0677] Step 3:

[0678] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[0679] Step 4:

[0680] The device receives the generated interactive art data from the server. The device stores this data in a buffer and performs initial setup to display the art in real time. It also prepares settings to receive data from the emotion engine and sensors.

[0681] Step 5:

[0682] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, by measuring the user's distance and type of action (e.g., waving, jumping, etc.).

[0683] Step 6:

[0684] The emotion engine analyzes the user's facial expressions and recognizes their emotions. Specifically, it uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[0685] Step 7:

[0686] The sensor and emotion engine send measured data to the device in real time, and the device analyzes the received sensor data and emotion data to identify the user's actions and emotions.

[0687] Step 8:

[0688] The device dynamically readjusts the generated interactive art based on data from sensors and an emotion engine, for example, changing the color of the art when the user approaches, or turning brighter if the user smiles.

[0689] Step 9:

[0690] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[0691] Step 10:

[0692] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[0693] Step 11:

[0694] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[0695] Step 12:

[0696] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[0697] Step 13:

[0698] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[0699] Example 2

[0700] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] In systems that project interactive art in public spaces, there is a need to provide a richer art experience by reflecting not only the movements but also the emotions of passersby. Another important issue is to contribute to the revitalization of the local area by incorporating local history and culture.

[0702] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation AI means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means for detecting the movements of passersby; emotion recognition means including an emotion engine that recognizes the emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion recognition means and projecting it in a public place; and collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation AI means. This makes it possible to generate interactive art that changes color and shape based on the movements and emotions of passersby and is used in local events and promotional activities.

[0703] "Artist-drawn design" means design data created by an artist that includes visual expressions and interactive elements.

[0704] "Generative AI means" is an artificial intelligence technology that analyzes the artist's design and generates interactive elements that change in response to the movements and emotions of passersby.

[0705] "Terminal means" refers to a device that receives interactive art data generated from the server and prepares it for real-time display and projection.

[0706] "Sensor means" refers to equipment or technology used to detect the movement of passersby.

[0707] The "emotion recognition means" is an emotion engine that includes face recognition technology to recognize the emotions of passersby.

[0708] "Projection means" means a device for projecting adjusted art data into a public space.

[0709] "Collaboration means" refers to mechanisms or means for collecting data on local historical elements, characters, etc., and providing it to the generative AI means.

[0710] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[0711] The server retrieves the artist's design from the database. The retrieved design includes visual and interactive elements. The server sends this design data to the generation AI, instructing it to generate interactive elements that change according to the movements and emotions of passersby. For example, the server retrieves the design data from a MySQL database and sends a POST request to the generation AI's API.

[0712] The generative AI then analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change in response to the movements and emotions of passersby. For example, it includes instructions such as "change color when a user approaches." This data is sent to the device via the server.

[0713] The device receives the generated interactive art data from the server and stores it in a buffer. The device prepares to receive data from sensors and emotion engines in real time. For example, the device receives art data from an AWS EC2 server and stores it in local storage.

[0714] When a user (passerby) passes through a public space, a sensor detects their movement. The data captured by the sensor is sent to the device in real time. At the same time, an emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger. For example, Microsoft Kinect is used as the sensor and Affectiva SDK is used as the emotion engine.

[0715] The device receives and analyzes motion data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. This adjusted art data is then sent to the projection device.

[0716] The projection device projects the adjusted interactive art data in high resolution into a public space. The projector continues to display the art while waiting for the next sensor and emotion data. For example, an EPSON projector is used to project the art.

[0717] Another feature of this system is its collaboration with local elements. The server collects data on the region's history, culture, popular characters, and other topics, and sends it to the AI ​​generator. The AI ​​analyzes this data and generates new interactive art. This art data is then sent back to the device via the server and used in local promotional events.

[0718] An example of a prompt is, "Using design data themed on the city of Shibuya, generate interactive art in which the scenery changes from night to day when passersby approach, and the colors change vividly when passersby smile." By inputting this prompt into a generative AI model, specific art is generated.

[0719] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0720] Step 1:

[0721] The server retrieves the artist's design. Specifically, the server connects to a database (e.g., MySQL) and executes the query "SELECT FROM designs WHERE artist_id = '12345'". The input is the database connection information and the query, and the output is the retrieved design data. This design data includes visual and interactive elements.

[0722] Step 2:

[0723] The server sends the acquired design data to the generative AI. Specifically, the server sends an HTTP POST request to the API endpoint of the generative AI model, including the query result design data in JSON format. The input is the acquired design data, and the output is interactive art data generated by the generative AI model.

[0724] Step 3:

[0725] The generative AI analyzes the design data received from the server and generates interactive art. In this process, the generative AI analyzes the visual and interactive elements contained in the design data and generates art that changes in response to movement and emotions. The input is the design data sent from the server, and the output is the generated interactive art data.

[0726] Step 4:

[0727] The device receives the generated interactive art data from the server and stores it in a buffer. Specifically, the device retrieves the data from the server via an HTTP GET request and stores it in local storage (e.g., / tmp / art_data.json). The input is the interactive art data returned by the generation AI, and the output is the stored art data.

[0728] Step 5:

[0729] When a user (passerby) passes through a public space, a sensor detects their movement. Specifically, the sensor (e.g., Microsoft Kinect) captures the user's movement and transmits the data to a device in real time. The input is the user's movement, and the output is the detected movement data.

[0730] Step 6:

[0731] The emotion engine analyzes the user's facial expressions and recognizes emotions. Specifically, the emotion engine (e.g., Affectiva SDK) captures the user's face in real time and generates emotion data (happiness, surprise, sadness, anger, etc.). The input is the user's facial expression, and the output is the recognized emotion data.

[0732] Step 7:

[0733] The device receives and analyzes sensor movement data and emotion data from the emotion engine. Specifically, the device analyzes the data received in real time and dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. The input is data from the sensor and emotion engine, and the output is the adjusted art data.

[0734] Step 8:

[0735] The terminal transmits the adjusted art data to a projection device, which projects it into a public space. Specifically, the terminal transmits the art data to a projector (e.g., an EPSON projector) and projects it in high resolution. The input is the adjusted interactive art data, and the output is the art projected into a public space.

[0736] Step 9:

[0737] The server collects regional data (history, characters, etc.) and sends it to the generation AI. Specifically, the server collects regional data from APIs and databases and sends it to the generation AI. The input is regional data, and the output is the generated interactive art specific to the region.

[0738] Step 10:

[0739] Interactive art specific to the region is generated and sent to the device. Specifically, the generation AI analyzes the region data received from the server, generates new interactive art, and sends it to the device. The input is the region data from the server, and the output is the generated interactive art data specific to the region.

[0740] (Application example 2)

[0741] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0742] Currently, public spaces and commercial facilities mainly present static advertisements and art exhibits to passersby and customers, lacking in interactive experiences. Furthermore, there has been little progress in practical application of technology that provides dynamic content that reflects the emotions of passersby and customers. This means that traditional advertisements and exhibits are one-way, making it difficult to attract users' attention. Furthermore, there is a lack of content that reflects local culture and history, making it difficult to contribute to regional revitalization.

[0743] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means and emotion engine means for detecting the movements and emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion engine and projecting it in a public place or commercial facility; and collaboration means for collecting local cultural elements and providing them to the generation means. This makes it possible to provide passersby and customers with an interactive, dynamic art experience that responds to their emotions, and to contribute to regional revitalization with content that reflects the local culture and history.

[0744] "Means for capturing artist-drawn designs" refers to any equipment or software used to receive or capture in digital form the visual designs created by the artist.

[0745] "Generation means" refers to software or algorithms that generate interactive elements based on acquired design data.

[0746] "Terminal means" refers to a process or device for receiving, displaying, and projecting the generated art data.

[0747] "Sensor means" refers to devices or technologies for detecting the movement of users or passersby.

[0748] "Emotion engine means" refers to software or algorithms that analyze the facial expressions of users and passersby and identify their emotions.

[0749] "Projection means" refers to a projector or related device for projecting art data onto a wall or other surface in a public place or commercial facility.

[0750] "Collaboration tools" refer to databases and software systems that collect local cultural elements and provide them to production tools.

[0751] MODE FOR CARRYING OUT THE INVENTION

[0752] Overall system configuration

[0753] This invention is a system for projecting interactive art in public places and commercial facilities based on designs drawn by artists. This system is composed of a generation means for analyzing and generating acquired design data, a sensor means and emotion engine means for detecting movement and emotions, a terminal means for receiving and displaying data, a projection means for projecting the art, and a collaboration means for collecting local cultural elements.

[0754] Specific examples of each component

[0755] server

[0756] The server retrieves and analyzes the artist's design. The design data includes tags and metadata for adding interactive elements. The server then sends this design data to the AI, instructing it to generate interactive art based on the movements and emotions of passersby.

[0757] generation means

[0758] The generation method uses a generative AI model (e.g., OpenAI's DALL-E or GPT-4), analyzes the design data received from the server, and generates interactive art that changes in response to the user's movements and emotions.

[0759] Terminal means

[0760] The terminal means receives the generated interactive art data, stores the data in a buffer, performs initialization for displaying the art in real time, and prepares to receive data from the sensor means and emotion engine means in real time.

[0761] Sensor Means and Emotion Engine Means

[0762] The sensor means uses cameras and IR sensors, which detect the movements of users (passersby) in real time. The emotion engine means uses Microsoft Azure Cognitive Services API to analyze facial expressions and identify emotions such as joy, surprise, sadness, and anger.

[0763] Projection Method

[0764] A high-resolution projector is used as a projection medium, and the generated interactive art data is projected into public spaces or commercial facilities.

[0765] Collaboration tools

[0766] The server collects local cultural elements and provides them to the generation means, allowing the generation AI to generate interactive art that incorporates new elements.

[0767] Example

[0768] Specific examples include the following:

[0769] Interactive art in public spaces

[0770] The server retrieves designs that reflect the characteristics of the area, and the AI ​​generates interactive art that changes in response to the movements and emotions of passersby.

[0771] Example prompt sentence:

[0772] Generate an art piece that changes color and form based on user emotions and movements detected via sensors.

[0773] The device receives data from the sensors and configures it to display art in real time.

[0774] When passersby smile, the projected art changes into vibrant colors and adds animation effects.

[0775] Art exhibition in a commercial facility

[0776] The server retrieves designs based on the theme of the commercial facility and generates interactive art using generative AI.

[0777] The device displays art on its display and receives data from sensors and an emotion engine.

[0778] As customers pass by the display in the store, the art changes based on their movements and facial expressions, providing a more enjoyable shopping experience.

[0779] In this way, the system implementing the invention can effectively combine an improved art experience with local cultural elements.

[0780] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0781] Step 1:

[0782] The server retrieves the artist's design from the database. When retrieving the design data, it also includes tags and metadata for adding interactive elements. This becomes the input data for the server. The retrieved design data is then analyzed in digital format and processed into a format that can be sent to the generation AI. This processed data becomes the output data from the server to the generation AI.

[0783] Step 2:

[0784] The server sends the acquired design data to the generation AI, which then generates interactive art based on a prompt (e.g., "Generate an art piece that changes color and form based on user emotions and movements detected via sensors."). The generation AI analyzes the data and generates art, and the generated interactive art data is the output.

[0785] Step 3:

[0786] The server receives the interactive art data generated by the generation AI and sends it to the terminal. The terminal stores the received data in a buffer and performs initial settings to display the art in real time. This becomes input data to the terminal. The terminal prepares to receive data from the sensor means and emotion engine means in real time.

[0787] Step 4:

[0788] When a user (passerby) visits a public place or commercial facility, the sensor means detects the user's movements, and at the same time the emotion engine means analyzes the user's facial expressions. The movement data and emotion data acquired by the sensor and emotion engine become input data to the terminal. Based on this data, the terminal performs data analysis and dynamically adjusts the art data.

[0789] Step 5:

[0790] The device adjusts the interactive art data in real time based on the motion and emotion data. For example, it can change the color or shape of the art when the user approaches, or change to a brighter color tone when the user smiles. This adjusted art data becomes input data for the projection means.

[0791] Step 6:

[0792] The device then transmits the adjusted interactive art data to a projector, which projects it into a public or commercial space. The projector displays the art in high resolution and waits for new sensor and emotion data input. The displayed art allows users to enjoy an interactive experience.

[0793] Step 7:

[0794] The server collects local cultural and historical elements from a database and provides this data to a generation AI, which then generates new interactive art incorporating elements unique to the region or commercial facility. This newly generated art data becomes output data for the server and can be used again on devices or projection means.

[0795] This process brings together the server, the device, and the user to create a dynamic interactive art experience.

[0796] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0797] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0798] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0799] [Third embodiment]

[0800] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0801] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0802] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0803] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0804] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0805] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0806] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0807] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0808] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0809] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0810] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0811] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0812] This invention is a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists. The system's main components are a server, a terminal, and a user.

[0813] System Overview

[0814] 1. The server acquires and analyzes the artist's design

[0815] The first step the server takes is to retrieve the artist's design from the database. This design is visually represented and includes tags and metadata. The server then asks the generation AI to analyze the retrieved design. Once the generation AI begins its analysis, it extracts each element in the design and adds interactive elements (for example, parts that change color depending on how close passersby are).

[0816] 2. Generating Interactive Art with Generative AI

[0817] The AI ​​analyzes the design received from the server and begins generating interactive art. The generated art has dynamic elements, such as changing color or shape when passersby approach. The generated interactive art data is then returned to the server.

[0818] 3. Data reception by the terminal and preparation for projection

[0819] The server sends the generated interactive art data to the device. The device stores the received art data in a buffer and prepares for projection. At this time, the device checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest art data.

[0820] 4. Detecting and Processing User Movements

[0821] When a user (passerby) moves through a public space, sensors detect their movements. The data measured by the sensors is sent to the device in real time. The sensors accurately measure the passerby's distance and type of movement (e.g., waving, jumping, etc.). The device receives this data and dynamically adjusts the generated art data and projects it into the public space.

[0822] 5. Collaboration that combines local elements

[0823] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[0824] Specific examples

[0825] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[0826] Server: Initial setup and art design acquisition

[0827] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[0828] Generative AI: Generating Interactive Art

[0829] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically in response to the movements of passersby.

[0830] Terminal: Receiving art data and preparing for projection

[0831] The generated art is sent to the device, which then prepares it for projection onto the wall. The device monitors the movements of passersby in real time via sensors.

[0832] User (passerby): Motion detection and art changes

[0833] Sensors detect the movements of passersby and send that data to a device. For example, if a passerby approaches the wall and waves, the art changes color and the scenery changes accordingly.

[0834] Collaboration with the local community

[0835] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[0836] This system will provide passersby with a new art experience, while also drawing out the charm of the local area and promoting its revitalization.

[0837] The processing flow will be explained below.

[0838] Step 1:

[0839] The server retrieves the artist's design from the database. This is a process of searching for design data using the specified artist ID or project ID and downloading the necessary files.

[0840] Step 2:

[0841] The server sends the acquired design data to the generation AI, which includes tags and metadata for adding interactive elements. The generation AI analyzes the design data and generates interactive elements (e.g., color changes, shape changes) that change in response to the movements of passersby.

[0842] Step 3:

[0843] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[0844] Step 4:

[0845] The device receives the generated interactive art data from the server, stores the data in a buffer, and performs initialization to display the art in real time.

[0846] Step 5:

[0847] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, measuring the user's distance and type of movement (e.g., waving, jumping, etc.).

[0848] Step 6:

[0849] The sensor data is sent to the device in real time, and the device analyzes the received sensor data to identify the user's actions.

[0850] Step 7:

[0851] The device dynamically readjusts the generated interactive art based on data from sensors: for example, the art changes color when the user approaches, or rotates when the user waves their hand.

[0852] Step 8:

[0853] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a wall or other public space. The projector displays the art in high resolution.

[0854] Step 9:

[0855] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[0856] Step 10:

[0857] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[0858] Step 11:

[0859] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[0860] Step 12:

[0861] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[0862] Example 1

[0863] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0864] Traditional art exhibitions and installations have often been static, with limited interaction with the audience. Furthermore, there has been a lack of efforts to create dynamic art that incorporates the characteristics and culture of the local area. This has limited the appeal of art to audiences and placed limitations on local promotional activities. As a result, there has been a demand for ways to strengthen the connections between art, audiences, and the local area.

[0865] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0866] In this invention, the server includes a means for acquiring designs, a generating means for analyzing the acquired designs and generating elements that change in response to movement, a terminal means for receiving and preparing the generated interactive art data, a sensor means for detecting movement, a projection means for dynamically adjusting the art data based on data from the sensor and projecting it in a public space, and a linking means for collecting data on local historical elements, characters, etc. and providing it to the generating means. This enables interactive art exhibitions, where art changes in real time in response to the movement of visitors, strengthening ties with the local community. It can also provide a new art experience for local promotional activities.

[0867] A "design" refers to an artist's visual representation or creation, including tags and metadata.

[0868] "Generation means" refers to a means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby or spectators.

[0869] "Terminal means" refers to a device or system that receives the generated interactive art data, stores it, and prepares it for projection.

[0870] "Sensor means" refers to devices or systems that detect the movements of passersby and spectators and collect that data in real time.

[0871] "Projection Measure" refers to a device or system for projecting dynamically adjusted artistic data into a public space based on data from sensors.

[0872] "Collaboration means" refers to devices or systems that collect data such as historical elements and characters of a region, provide it to the generation means, and use it to generate interactive art.

[0873] "Public place" refers to any outdoor or indoor space that is freely accessible to passersby and spectators.

[0874] This invention is a system that acquires designs drawn by artists, analyzes them using a generative AI model, and generates and projects interactive art. The system's main components are a server, a terminal, and a user.

[0875] Overall system picture

[0876] This system is designed to provide interactive art experiences in public spaces. The main hardware used includes a database management system, servers, sensors, high-performance PCs, and projectors. The software used includes generative AI models and libraries for real-time processing. Specifically, the database management system uses MySQL or PostgreSQL, the servers use Amazon Web Services (AWS) or Google Cloud Platform (GCP), the generative AI models use OpenAI GPT-4 or DALL-E 2, and the real-time processing uses OpenCV.

[0877] Server acquires and analyzes the design

[0878] The server first retrieves the artist's design from a database. This design contains tags and metadata, and the generative AI model analyzes it based on this information. Once the analysis begins, the generative AI extracts each element in the design and adds interactive features, such as elements that change color or shape when passersby approach.

[0879] Generating interactive art using generative AI models

[0880] The generative AI model generates interactive art based on the data received from the server. Specifically, a prompt is provided to the generative AI model, which then generates art with dynamic elements based on that prompt.

[0881] Data reception by terminal and preparation for projection

[0882] The interactive art data generated from the server is sent to the device. The device receives this data and stores it in a buffer. It then prepares for projection and checks and initializes the sensor settings. This ensures that the device is always ready to project the latest art data.

[0883] User movement detection and processing

[0884] When users (passersby) move around in public spaces, sensors detect their movements in real time. The sensors accurately measure the distance and actions of passersby (such as waving or jumping) and send that data to the device. The device then uses this data to dynamically adjust and project the generated art.

[0885] Collaboration with the local community

[0886] The server collects data on historical elements and popular characters of the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines this data with the art to generate new interactive art. This data is then sent back to the device via the server and used in local promotional activities and events.

[0887] Examples of specific examples and prompts

[0888] As a specific scenario, consider the projection of new art in Shibuya, Tokyo. The server acquires Shibuya-themed art designs and has the generation AI analyze them. The generation AI generates interactive art that changes the time of day from night to day when passersby approach. This data is sent to the device and projected by the projector. Sensors detect the movement of passersby, causing the art to change dynamically.

[0889] Prompt Sentence Examples

[0890] "Analyze an art design containing iconic Shibuya scenery and generate interactive art that changes the time of day from night to day when passersby approach."

[0891] This system will provide passersby with a new art experience, draw out the charm of the area, and promote revitalization.

[0892] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0893] Program processing steps

[0894] Step 1: Obtain the design and request analysis

[0895] input

[0896] Design data in the database

[0897] output

[0898] Analysis prompts and design information sent to the generative AI

[0899] Specific actions

[0900] The server retrieves the artist's designs from the database, which contain tags and metadata, analyzes the designs, and prepares to send prompts to the generation AI.

[0901] sql

[0902] SELECT FROM designs WHERE artist_id = 'artist123';

[0903] Prompt Sentence Examples

[0904] "Analyze a design containing an iconic view of Shibuya and generate interactive art that changes the time of day from night to day as passersby approach."

[0905] Step 2: Generative AI creates interactive art

[0906] input

[0907] Prompts and design information sent from the server

[0908] output

[0909] Generated interactive art data

[0910] Specific actions

[0911] The generative AI takes design data received from the server and generates interactive art based on specified prompts, such as programming elements in the design to change in response to movement.

[0912] python

[0913] prompt = "A design featuring the iconic Shibuya scenery, with added details that change based on the movement of passersby."

[0914] ai_response = gpt4.generate(prompt)

[0915] Step 3: Receiving and preparing the art data on your device

[0916] input

[0917] Interactive art data generated by generative AI

[0918] output

[0919] Art data stored in the buffer, initial sensor settings

[0920] Specific actions

[0921] The server sends the generated interactive art data to the device using a REST API. The device stores the received data in a buffer, prepares for projection, and checks and initializes the sensor settings.

[0922] python

[0923] response = requests.post('http: / / endpoint_to_terminal', json=art_data)

[0924] with open('art_data_buffer.bin', 'wb') as buffer_file:

[0925] buffer_file.write(response.content)

[0926] Step 4: Detecting user movements and processing data

[0927] input

[0928] User movement data detected by sensors (e.g., distance, type of movement)

[0929] output

[0930] Dynamically adjusted art data

[0931] Specific actions

[0932] As users move through public spaces, sensors detect their movements in real time, measuring the distance and type of movement of passersby and transmitting that data to the device, which then dynamically adjusts and projects the generated art.

[0933] python

[0934] sensor_data = sensor.read_data()

[0935] if sensor_data['distance'] < threshold:

[0936] update_art_display('interaction_type', 'color_change')

[0937] Step 5: Collect and fuse local data

[0938] input

[0939] Regional historical elements and character data

[0940] output

[0941] New prompts and design data provided to the generative AI

[0942] Specific actions

[0943] The server collects data on local historical elements and popular characters, and instructs the AI ​​to analyze it. The AI ​​then combines this data into interactive art to generate new art data, which can be used in local events and promotional activities.

[0944] sql

[0945] SELECT FROM regional_data WHERE region = 'Shibuya';

[0946] Prompt Sentence Examples

[0947] "Create interactive art that includes historical elements and famous characters from Shibuya."

[0948] This will enable interactive art exhibitions, allowing the art to change in real time in response to the movements of visitors, and will also strengthen ties with the local community.

[0949] (Application example 1)

[0950] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0951] In modern public facilities and commercial areas, static decorations and advertisements do not have enough impact on passersby, making it difficult to increase customer engagement. It is also difficult to bring out the unique charm of a region without utilizing its historical elements and character. Furthermore, it is necessary to provide a more engaging experience by incorporating interactive elements that recognize passersby's movements and behavior in real time and dynamically change accordingly. A new system is needed to solve these issues, increase customer engagement, and improve the appeal of commercial areas and public facilities.

[0952] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0953] In this invention, the server includes means for acquiring designs drawn by artists, means for analyzing the acquired designs and generating interactive elements that change in response to the movements of passersby, terminal means for receiving and preparing the generated interactive art data, sensor means for detecting the movements of passersby, projection means for dynamically adjusting the art data based on the data from the sensors and projecting it in a public space, collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation means, implementation means for projecting the generated interactive art as a decorative display inside the store, and means for increasing customer engagement through the interactive art. This makes it possible to provide a more attractive and interactive experience and increase customer engagement compared to static decorations or advertisements.

[0954] An "artist-drawn design" is a work of art created by an artist, expressed as a visual element and including tags and metadata.

[0955] "Means for obtaining" refers to the methods and technical devices for obtaining the artist-drawn designs from the database.

[0956] "Analysis and generation of interactive elements that change in response to the movements of passersby" refers to the method and technical device for analyzing an artist's design using generative AI and generating interactive art that includes dynamic elements that change in response to the movements of passersby.

[0957] "Terminal means for receiving and preparing generated interactive art data" refers to a terminal device that receives interactive art data sent from the server and prepares it for projection.

[0958] "Sensor means for detecting the movement of passersby" refers to various sensors and their systems for detecting the distance and movement of passersby in real time.

[0959] "Projection means for dynamically adjusting art data based on data from sensors and projecting it into a public space" refers to a method and technical device for updating interactive art data based on data acquired by sensors and using a projector to project art into a public space.

[0960] "Collaboration means for collecting data on regional historical elements, characters, etc. and providing it to the generation means" refers to the methods and technical devices for collecting historical aspects and character information related to a specific region and providing it to the generation AI.

[0961] "Interior store decorative displays" refer to devices that are installed on walls, floors, etc. inside commercial facilities to display designs and information.

[0962] "Means to increase customer engagement" refers to methods and technological devices that increase customer relationships and interest through interactive experiences and art.

[0963] This invention is a system that enhances customer engagement in brick-and-mortar stores through interactive art based on artist designs. The system is primarily comprised of a server, terminals, and users, each of which plays a role in generating and projecting interactive art.

[0964] Server Roles

[0965] The server first retrieves the artist's design from the database. This design is visually represented and includes tags and metadata as additional information. The server then passes the retrieved design to a generation AI, which analyzes each element in the design and generates interactive art. The generated interactive art data is then sent from the server to the device.

[0966] Device Role

[0967] The device receives the interactive art data sent from the server and stores it in a buffer. The device prepares for projection and also checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest interactive art.

[0968] User Roles

[0969] As users (customers) move around the store, sensors detect their movements. The sensors accurately measure the user's distance and actions (e.g., waving, jumping, etc.). The device that receives this data dynamically adjusts the generated art data and projects it onto the walls and floors inside the store.

[0970] Hardware and Software Details

[0971] Hardware

[0972] Cameras and sensors: to detect user movement and distance

[0973] Projector: for projecting interactive art

[0974] software

[0975] OpenCV: for image processing and display

[0976] TensorFlow / Keras: for using generative AI models

[0977] The server retrieves the artist's design and uses generative AI to analyze and generate interactive art. The generated art data is adjusted in real time based on customer movements detected by sensors and projected inside the store via a projector. This provides a more engaging and interactive experience than static decorations or advertisements, and can improve customer engagement.

[0978] Specific examples and prompts for the generative AI model

[0979] Specific examples

[0980] Art projected onto the walls of the shopping mall changes into a flower design when passersby approach, and it also recognizes certain actions (e.g. jumping) and displays special effects according to the action.

[0981] Example prompts

[0982] Artist Design: Cityscape

[0983] Effect: The city changes from day to night as passersby approach

[0984] Interaction: Buildings change color when passersby wave

[0985] Regional elements: Designs incorporating regional history and famous characters

[0986] This example shows how store décor can go beyond being just a visual element to provide customers with a dynamic and interactive experience, capturing their attention, increasing their dwell time in the store, and increasing their engagement.

[0987] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0988] Step 1:

[0989] The server retrieves the artist's drawing from the database.

[0990] Input: Artist's design ID

[0991] Data processing: Search the database based on the design ID and obtain the corresponding design data

[0992] Output: Artist's design (image file and additional information)

[0993] Step 2:

[0994] The server passes the acquired design to the generative AI model and begins analysis.

[0995] Input: Artist's design (image file and additional information)

[0996] Data Computation: Generative AI models analyze the visual elements of a design and generate dynamic, interactive elements.

[0997] Output: Interactive art data

[0998] Step 3:

[0999] The server transmits the generated interactive art data to the terminal.

[1000] Input: Interactive art data

[1001] Data processing: Converting interactive art data into a format that can be received by the device

[1002] Output: Converted interactive art data

[1003] Step 4:

[1004] The terminal stores the received interactive art data in a buffer and prepares it for projection.

[1005] Input: Generated interactive art data

[1006] Data processing: Save interactive art data to a buffer, check and initialize projector settings

[1007] Output: The projector is ready to project.

[1008] Step 5:

[1009] As users (customers) move around the store, sensors detect their movements.

[1010] Input: User movement (distance and action)

[1011] Data processing: Motion detected by sensors is converted into data in real time

[1012] Output: Detected motion data

[1013] Step 6:

[1014] The device dynamically adjusts the art data based on data from the sensors and performs the projection.

[1015] Input: User movement data from sensors, interactive art data

[1016] Data Calculation: Dynamically update the color and shape of interactive art based on user movement

[1017] Output: Updated interactive art (projection)

[1018] Step 7:

[1019] The server collects data on the region's historical elements, characters, etc. and provides it to the generative AI model.

[1020] Input: Regional historical data, character data

[1021] Data processing: Converting collected data into a format that can be analyzed by the AI ​​model

[1022] Output: Interactive art data including regional data

[1023] Step 8:

[1024] The terminal then projects the final interactive art as a decorative display inside the store.

[1025] Input: Final generated interactive art data

[1026] Data processing: Projecting interactive art data optimized for display

[1027] Output: Interactive art projected onto decorative displays inside the store

[1028] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1029] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[1030] System Overview

[1031] 1. The server acquires and analyzes the artist's design

[1032] The first step the server takes is to retrieve the artist's design from the database. The retrieved design includes the artist's intended visual expression as well as tags and metadata for adding interactive elements. The server then sends the retrieved design data to the generation AI, instructing it to generate interactive elements that change in response to the movements and emotions of passersby.

[1033] 2. Generating Interactive Art with Generative AI

[1034] The generative AI analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change according to the movements and emotions of passersby. This data is then sent to the device via the server.

[1035] 3. Data reception by the terminal and preparation for projection

[1036] The device receives the generated interactive art data from the server, stores this data in a buffer, and initializes the device to display the art in real time. Additionally, the device prepares to receive data from the sensors and emotion engine in real time.

[1037] 4. User movement detection and emotion recognition

[1038] When a user (passerby) passes through a public space, a sensor detects their movement. The data measured by the sensor is sent to the device in real time. At the same time, the emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[1039] 5. Device processing of sensor and emotion data

[1040] The device receives and analyzes movement data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, the color and shape of the art may change when the user approaches, or the color may become brighter if the user smiles.

[1041] 6. Projecting art using projection media

[1042] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[1043] 7. Collaboration that combines local elements

[1044] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[1045] Specific examples

[1046] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[1047] Server: Initial setup and art design acquisition

[1048] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[1049] Generative AI: Generating Interactive Art

[1050] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically according to the movements and emotions of passersby.

[1051] Terminal: Receiving art data and preparing for projection

[1052] The generated art is sent to the device, which prepares it for projection onto the wall. The device monitors passersby's movements via sensors and their emotions via an emotion engine in real time.

[1053] User (passerby): Motion and emotion detection, and art changes

[1054] Sensors detect the movements of passersby, and an emotion engine analyzes their facial expressions to identify their emotions. This data is then sent to the device. For example, if a passerby approaches the wall and smiles, the colors of the artwork will become more vibrant and the scenery will change to reflect their emotions.

[1055] Collaboration with the local community

[1056] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[1057] This system provides passersby with a deeper art experience, drawing out the local area's charm and promoting its revitalization. In addition, by combining it with an emotion engine, it becomes possible to provide individual experiences that correspond to the user's emotions.

[1058] The processing flow will be explained below.

[1059] Step 1:

[1060] The server retrieves the artist's design from the database. Specifically, it searches for the design data using the specified artist ID or project ID and downloads the necessary files.

[1061] Step 2:

[1062] The server sends the acquired design data, which includes tags and metadata for adding interactive elements, to the generation AI, which analyzes the design data and generates interactive elements that change in response to the movements and emotions of passersby.

[1063] Step 3:

[1064] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[1065] Step 4:

[1066] The device receives the generated interactive art data from the server. The device stores this data in a buffer and performs initial setup to display the art in real time. It also prepares settings to receive data from the emotion engine and sensors.

[1067] Step 5:

[1068] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, by measuring the user's distance and type of action (e.g., waving, jumping, etc.).

[1069] Step 6:

[1070] The emotion engine analyzes the user's facial expressions and recognizes their emotions. Specifically, it uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[1071] Step 7:

[1072] The sensor and emotion engine send measured data to the device in real time, and the device analyzes the received sensor data and emotion data to identify the user's actions and emotions.

[1073] Step 8:

[1074] The device dynamically readjusts the generated interactive art based on data from sensors and an emotion engine, for example, changing the color of the art when the user approaches, or turning brighter if the user smiles.

[1075] Step 9:

[1076] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[1077] Step 10:

[1078] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[1079] Step 11:

[1080] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[1081] Step 12:

[1082] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[1083] Step 13:

[1084] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[1085] Example 2

[1086] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1087] In systems that project interactive art in public spaces, there is a need to provide a richer art experience by reflecting not only the movements but also the emotions of passersby. Another important issue is to contribute to the revitalization of the local area by incorporating local history and culture.

[1088] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation AI means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means for detecting the movements of passersby; emotion recognition means including an emotion engine that recognizes the emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion recognition means and projecting it in a public place; and collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation AI means. This makes it possible to generate interactive art that changes color and shape based on the movements and emotions of passersby and is used in local events and promotional activities.

[1089] "Artist-drawn design" means design data created by an artist that includes visual expressions and interactive elements.

[1090] "Generative AI means" is an artificial intelligence technology that analyzes the artist's design and generates interactive elements that change in response to the movements and emotions of passersby.

[1091] "Terminal means" refers to a device that receives interactive art data generated from the server and prepares it for real-time display and projection.

[1092] "Sensor means" refers to equipment or technology used to detect the movement of passersby.

[1093] The "emotion recognition means" is an emotion engine that includes face recognition technology to recognize the emotions of passersby.

[1094] "Projection means" means a device for projecting adjusted art data into a public space.

[1095] "Collaboration means" refers to mechanisms or means for collecting data on local historical elements, characters, etc., and providing it to the generative AI means.

[1096] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[1097] The server retrieves the artist's design from the database. The retrieved design includes visual and interactive elements. The server sends this design data to the generation AI, instructing it to generate interactive elements that change according to the movements and emotions of passersby. For example, the server retrieves the design data from a MySQL database and sends a POST request to the generation AI's API.

[1098] The generative AI then analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change in response to the movements and emotions of passersby. For example, it includes instructions such as "change color when a user approaches." This data is sent to the device via the server.

[1099] The device receives the generated interactive art data from the server and stores it in a buffer. The device prepares to receive data from sensors and emotion engines in real time. For example, the device receives art data from an AWS EC2 server and stores it in local storage.

[1100] When a user (passerby) passes through a public space, a sensor detects their movement. The data captured by the sensor is sent to the device in real time. At the same time, an emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger. For example, Microsoft Kinect is used as the sensor and Affectiva SDK is used as the emotion engine.

[1101] The device receives and analyzes motion data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. This adjusted art data is then sent to the projection device.

[1102] The projection device projects the adjusted interactive art data in high resolution into a public space. The projector continues to display the art while waiting for the next sensor and emotion data. For example, an EPSON projector is used to project the art.

[1103] Another feature of this system is its collaboration with local elements. The server collects data on the region's history, culture, popular characters, and other topics, and sends it to the AI ​​generator. The AI ​​analyzes this data and generates new interactive art. This art data is then sent back to the device via the server and used in local promotional events.

[1104] An example of a prompt is, "Using design data themed on the city of Shibuya, generate interactive art in which the scenery changes from night to day when passersby approach, and the colors change vividly when passersby smile." By inputting this prompt into a generative AI model, specific art is generated.

[1105] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1106] Step 1:

[1107] The server retrieves the artist's design. Specifically, the server connects to a database (e.g., MySQL) and executes the query "SELECT FROM designs WHERE artist_id = '12345'". The input is the database connection information and the query, and the output is the retrieved design data. This design data includes visual and interactive elements.

[1108] Step 2:

[1109] The server sends the acquired design data to the generative AI. Specifically, the server sends an HTTP POST request to the API endpoint of the generative AI model, including the query result design data in JSON format. The input is the acquired design data, and the output is interactive art data generated by the generative AI model.

[1110] Step 3:

[1111] The generative AI analyzes the design data received from the server and generates interactive art. In this process, the generative AI analyzes the visual and interactive elements contained in the design data and generates art that changes in response to movement and emotions. The input is the design data sent from the server, and the output is the generated interactive art data.

[1112] Step 4:

[1113] The device receives the generated interactive art data from the server and stores it in a buffer. Specifically, the device retrieves the data from the server via an HTTP GET request and stores it in local storage (e.g., / tmp / art_data.json). The input is the interactive art data returned by the generation AI, and the output is the stored art data.

[1114] Step 5:

[1115] When a user (passerby) passes through a public space, a sensor detects their movement. Specifically, the sensor (e.g., Microsoft Kinect) captures the user's movement and transmits the data to a device in real time. The input is the user's movement, and the output is the detected movement data.

[1116] Step 6:

[1117] The emotion engine analyzes the user's facial expressions and recognizes emotions. Specifically, the emotion engine (e.g., Affectiva SDK) captures the user's face in real time and generates emotion data (happiness, surprise, sadness, anger, etc.). The input is the user's facial expression, and the output is the recognized emotion data.

[1118] Step 7:

[1119] The device receives and analyzes sensor movement data and emotion data from the emotion engine. Specifically, the device analyzes the data received in real time and dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. The input is data from the sensor and emotion engine, and the output is the adjusted art data.

[1120] Step 8:

[1121] The terminal transmits the adjusted art data to a projection device, which projects it into a public space. Specifically, the terminal transmits the art data to a projector (e.g., an EPSON projector) and projects it in high resolution. The input is the adjusted interactive art data, and the output is the art projected into a public space.

[1122] Step 9:

[1123] The server collects regional data (history, characters, etc.) and sends it to the generation AI. Specifically, the server collects regional data from APIs and databases and sends it to the generation AI. The input is regional data, and the output is the generated interactive art specific to the region.

[1124] Step 10:

[1125] Interactive art specific to the region is generated and sent to the device. Specifically, the generation AI analyzes the region data received from the server, generates new interactive art, and sends it to the device. The input is the region data from the server, and the output is the generated interactive art data specific to the region.

[1126] (Application example 2)

[1127] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1128] Currently, public spaces and commercial facilities mainly present static advertisements and art exhibits to passersby and customers, lacking in interactive experiences. Furthermore, there has been little progress in practical application of technology that provides dynamic content that reflects the emotions of passersby and customers. This means that traditional advertisements and exhibits are one-way, making it difficult to attract users' attention. Furthermore, there is a lack of content that reflects local culture and history, making it difficult to contribute to regional revitalization.

[1129] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means and emotion engine means for detecting the movements and emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion engine and projecting it in a public place or commercial facility; and collaboration means for collecting local cultural elements and providing them to the generation means. This makes it possible to provide passersby and customers with an interactive, dynamic art experience that responds to their emotions, and to contribute to regional revitalization with content that reflects the local culture and history.

[1130] "Means for capturing artist-drawn designs" refers to any equipment or software used to receive or capture in digital form the visual designs created by the artist.

[1131] "Generation means" refers to software or algorithms that generate interactive elements based on acquired design data.

[1132] "Terminal means" refers to a process or device for receiving, displaying, and projecting the generated art data.

[1133] "Sensor means" refers to devices or technologies for detecting the movement of users or passersby.

[1134] "Emotion engine means" refers to software or algorithms that analyze the facial expressions of users and passersby and identify their emotions.

[1135] "Projection means" refers to a projector or related device for projecting art data onto a wall or other surface in a public place or commercial facility.

[1136] "Collaboration tools" refer to databases and software systems that collect local cultural elements and provide them to production tools.

[1137] MODE FOR CARRYING OUT THE INVENTION

[1138] Overall system configuration

[1139] This invention is a system for projecting interactive art in public places and commercial facilities based on designs drawn by artists. This system is composed of a generation means for analyzing and generating acquired design data, a sensor means and emotion engine means for detecting movement and emotions, a terminal means for receiving and displaying data, a projection means for projecting the art, and a collaboration means for collecting local cultural elements.

[1140] Specific examples of each component

[1141] server

[1142] The server retrieves and analyzes the artist's design. The design data includes tags and metadata for adding interactive elements. The server then sends this design data to the AI, instructing it to generate interactive art based on the movements and emotions of passersby.

[1143] generation means

[1144] The generation method uses a generative AI model (e.g., OpenAI's DALL-E or GPT-4), analyzes the design data received from the server, and generates interactive art that changes in response to the user's movements and emotions.

[1145] Terminal means

[1146] The terminal means receives the generated interactive art data, stores the data in a buffer, performs initialization for displaying the art in real time, and prepares to receive data from the sensor means and emotion engine means in real time.

[1147] Sensor Means and Emotion Engine Means

[1148] The sensor means uses cameras and IR sensors, which detect the movements of users (passersby) in real time. The emotion engine means uses Microsoft Azure Cognitive Services API to analyze facial expressions and identify emotions such as joy, surprise, sadness, and anger.

[1149] Projection Method

[1150] A high-resolution projector is used as a projection medium, and the generated interactive art data is projected into public spaces or commercial facilities.

[1151] Collaboration tools

[1152] The server collects local cultural elements and provides them to the generation means, allowing the generation AI to generate interactive art that incorporates new elements.

[1153] Example

[1154] Specific examples include the following:

[1155] Interactive art in public spaces

[1156] The server retrieves designs that reflect the characteristics of the area, and the AI ​​generates interactive art that changes in response to the movements and emotions of passersby.

[1157] Example prompt sentence:

[1158] Generate an art piece that changes color and form based on user emotions and movements detected via sensors.

[1159] The device receives data from the sensors and configures it to display art in real time.

[1160] When passersby smile, the projected art changes into vibrant colors and adds animation effects.

[1161] Art exhibition in a commercial facility

[1162] The server retrieves designs based on the theme of the commercial facility and generates interactive art using generative AI.

[1163] The device displays art on its display and receives data from sensors and an emotion engine.

[1164] As customers pass by the display in the store, the art changes based on their movements and facial expressions, providing a more enjoyable shopping experience.

[1165] In this way, the system implementing the invention can effectively combine an improved art experience with local cultural elements.

[1166] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1167] Step 1:

[1168] The server retrieves the artist's design from the database. When retrieving the design data, it also includes tags and metadata for adding interactive elements. This becomes the input data for the server. The retrieved design data is then analyzed in digital format and processed into a format that can be sent to the generation AI. This processed data becomes the output data from the server to the generation AI.

[1169] Step 2:

[1170] The server sends the acquired design data to the generation AI, which then generates interactive art based on a prompt (e.g., "Generate an art piece that changes color and form based on user emotions and movements detected via sensors."). The generation AI analyzes the data and generates art, and the generated interactive art data is the output.

[1171] Step 3:

[1172] The server receives the interactive art data generated by the generation AI and sends it to the terminal. The terminal stores the received data in a buffer and performs initial settings to display the art in real time. This becomes input data to the terminal. The terminal prepares to receive data from the sensor means and emotion engine means in real time.

[1173] Step 4:

[1174] When a user (passerby) visits a public place or commercial facility, the sensor means detects the user's movements, and at the same time the emotion engine means analyzes the user's facial expressions. The movement data and emotion data acquired by the sensor and emotion engine become input data to the terminal. Based on this data, the terminal performs data analysis and dynamically adjusts the art data.

[1175] Step 5:

[1176] The device adjusts the interactive art data in real time based on the motion and emotion data. For example, it can change the color or shape of the art when the user approaches, or change to a brighter color tone when the user smiles. This adjusted art data becomes input data for the projection means.

[1177] Step 6:

[1178] The device then transmits the adjusted interactive art data to a projector, which projects it into a public or commercial space. The projector displays the art in high resolution and waits for new sensor and emotion data input. The displayed art allows users to enjoy an interactive experience.

[1179] Step 7:

[1180] The server collects local cultural and historical elements from a database and provides this data to a generation AI, which then generates new interactive art incorporating elements unique to the region or commercial facility. This newly generated art data becomes output data for the server and can be used again on devices or projection means.

[1181] This process brings together the server, the device, and the user to create a dynamic interactive art experience.

[1182] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1183] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1184] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1185] [Fourth embodiment]

[1186] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1187] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1188] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1189] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1190] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1191] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1192] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1193] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1194] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1195] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1196] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1197] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1198] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1199] This invention is a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists. The system's main components are a server, a terminal, and a user.

[1200] System Overview

[1201] 1. The server acquires and analyzes the artist's design

[1202] The first step the server takes is to retrieve the artist's design from the database. This design is visually represented and includes tags and metadata. The server then asks the generation AI to analyze the retrieved design. Once the generation AI begins its analysis, it extracts each element in the design and adds interactive elements (for example, parts that change color depending on how close passersby are).

[1203] 2. Generating Interactive Art with Generative AI

[1204] The AI ​​analyzes the design received from the server and begins generating interactive art. The generated art has dynamic elements, such as changing color or shape when passersby approach. The generated interactive art data is then returned to the server.

[1205] 3. Data reception by the terminal and preparation for projection

[1206] The server sends the generated interactive art data to the device. The device stores the received art data in a buffer and prepares for projection. At this time, the device checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest art data.

[1207] 4. Detecting and Processing User Movements

[1208] When a user (passerby) moves through a public space, sensors detect their movements. The data measured by the sensors is sent to the device in real time. The sensors accurately measure the passerby's distance and type of movement (e.g., waving, jumping, etc.). The device receives this data and dynamically adjusts the generated art data and projects it into the public space.

[1209] 5. Collaboration that combines local elements

[1210] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[1211] Specific examples

[1212] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[1213] Server: Initial setup and art design acquisition

[1214] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[1215] Generative AI: Generating Interactive Art

[1216] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically in response to the movements of passersby.

[1217] Terminal: Receiving art data and preparing for projection

[1218] The generated art is sent to the device, which then prepares it for projection onto the wall. The device monitors the movements of passersby in real time via sensors.

[1219] User (passerby): Motion detection and art changes

[1220] Sensors detect the movements of passersby and send that data to a device. For example, if a passerby approaches the wall and waves, the art changes color and the scenery changes accordingly.

[1221] Collaboration with the local community

[1222] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[1223] This system will provide passersby with a new art experience, while also drawing out the charm of the local area and promoting its revitalization.

[1224] The processing flow will be explained below.

[1225] Step 1:

[1226] The server retrieves the artist's design from the database. This is a process of searching for design data using the specified artist ID or project ID and downloading the necessary files.

[1227] Step 2:

[1228] The server sends the acquired design data to the generation AI, which includes tags and metadata for adding interactive elements. The generation AI analyzes the design data and generates interactive elements (e.g., color changes, shape changes) that change in response to the movements of passersby.

[1229] Step 3:

[1230] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[1231] Step 4:

[1232] The device receives the generated interactive art data from the server, stores the data in a buffer, and performs initialization to display the art in real time.

[1233] Step 5:

[1234] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, measuring the user's distance and type of movement (e.g., waving, jumping, etc.).

[1235] Step 6:

[1236] The sensor data is sent to the device in real time, and the device analyzes the received sensor data to identify the user's actions.

[1237] Step 7:

[1238] The device dynamically readjusts the generated interactive art based on data from sensors: for example, the art changes color when the user approaches, or rotates when the user waves their hand.

[1239] Step 8:

[1240] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a wall or other public space. The projector displays the art in high resolution.

[1241] Step 9:

[1242] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[1243] Step 10:

[1244] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[1245] Step 11:

[1246] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[1247] Step 12:

[1248] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[1249] Example 1

[1250] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1251] Traditional art exhibitions and installations have often been static, with limited interaction with the audience. Furthermore, there has been a lack of efforts to create dynamic art that incorporates the characteristics and culture of the local area. This has limited the appeal of art to audiences and placed limitations on local promotional activities. As a result, there has been a demand for ways to strengthen the connections between art, audiences, and the local area.

[1252] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1253] In this invention, the server includes a means for acquiring designs, a generating means for analyzing the acquired designs and generating elements that change in response to movement, a terminal means for receiving and preparing the generated interactive art data, a sensor means for detecting movement, a projection means for dynamically adjusting the art data based on data from the sensor and projecting it in a public space, and a linking means for collecting data on local historical elements, characters, etc. and providing it to the generating means. This enables interactive art exhibitions, where art changes in real time in response to the movement of visitors, strengthening ties with the local community. It can also provide a new art experience for local promotional activities.

[1254] A "design" refers to an artist's visual representation or creation, including tags and metadata.

[1255] "Generation means" refers to a means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby or spectators.

[1256] "Terminal means" refers to a device or system that receives the generated interactive art data, stores it, and prepares it for projection.

[1257] "Sensor means" refers to devices or systems that detect the movements of passersby and spectators and collect that data in real time.

[1258] "Projection Measure" refers to a device or system for projecting dynamically adjusted artistic data into a public space based on data from sensors.

[1259] "Collaboration means" refers to devices or systems that collect data such as historical elements and characters of a region, provide it to the generation means, and use it to generate interactive art.

[1260] "Public place" refers to any outdoor or indoor space that is freely accessible to passersby and spectators.

[1261] This invention is a system that acquires designs drawn by artists, analyzes them using a generative AI model, and generates and projects interactive art. The system's main components are a server, a terminal, and a user.

[1262] Overall system picture

[1263] This system is designed to provide interactive art experiences in public spaces. The main hardware used includes a database management system, servers, sensors, high-performance PCs, and projectors. The software used includes generative AI models and libraries for real-time processing. Specifically, the database management system uses MySQL or PostgreSQL, the servers use Amazon Web Services (AWS) or Google Cloud Platform (GCP), the generative AI models use OpenAI GPT-4 or DALL-E 2, and the real-time processing uses OpenCV.

[1264] Server acquires and analyzes the design

[1265] The server first retrieves the artist's design from a database. This design contains tags and metadata, and the generative AI model analyzes it based on this information. Once the analysis begins, the generative AI extracts each element in the design and adds interactive features, such as elements that change color or shape when passersby approach.

[1266] Generating interactive art using generative AI models

[1267] The generative AI model generates interactive art based on the data received from the server. Specifically, a prompt is provided to the generative AI model, which then generates art with dynamic elements based on that prompt.

[1268] Data reception by terminal and preparation for projection

[1269] The interactive art data generated from the server is sent to the device. The device receives this data and stores it in a buffer. It then prepares for projection and checks and initializes the sensor settings. This ensures that the device is always ready to project the latest art data.

[1270] User movement detection and processing

[1271] When users (passersby) move around in public spaces, sensors detect their movements in real time. The sensors accurately measure the distance and actions of passersby (such as waving or jumping) and send that data to the device. The device then uses this data to dynamically adjust and project the generated art.

[1272] Collaboration with the local community

[1273] The server collects data on historical elements and popular characters of the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines this data with the art to generate new interactive art. This data is then sent back to the device via the server and used in local promotional activities and events.

[1274] Examples of specific examples and prompts

[1275] As a specific scenario, consider the projection of new art in Shibuya, Tokyo. The server acquires Shibuya-themed art designs and has the generation AI analyze them. The generation AI generates interactive art that changes the time of day from night to day when passersby approach. This data is sent to the device and projected by the projector. Sensors detect the movement of passersby, causing the art to change dynamically.

[1276] Prompt Sentence Examples

[1277] "Analyze an art design containing iconic Shibuya scenery and generate interactive art that changes the time of day from night to day when passersby approach."

[1278] This system will provide passersby with a new art experience, draw out the charm of the area, and promote revitalization.

[1279] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1280] Program processing steps

[1281] Step 1: Obtain the design and request analysis

[1282] input

[1283] Design data in the database

[1284] output

[1285] Analysis prompts and design information sent to the generative AI

[1286] Specific actions

[1287] The server retrieves the artist's designs from the database, which contain tags and metadata, analyzes the designs, and prepares to send prompts to the generation AI.

[1288] sql

[1289] SELECT FROM designs WHERE artist_id = 'artist123';

[1290] Prompt Sentence Examples

[1291] "Analyze a design containing an iconic view of Shibuya and generate interactive art that changes the time of day from night to day as passersby approach."

[1292] Step 2: Generative AI creates interactive art

[1293] input

[1294] Prompts and design information sent from the server

[1295] output

[1296] Generated interactive art data

[1297] Specific actions

[1298] The generative AI takes design data received from the server and generates interactive art based on specified prompts, such as programming elements in the design to change in response to movement.

[1299] python

[1300] prompt = "A design featuring the iconic Shibuya scenery, with added details that change based on the movement of passersby."

[1301] ai_response = gpt4.generate(prompt)

[1302] Step 3: Receiving and preparing the art data on your device

[1303] input

[1304] Interactive art data generated by generative AI

[1305] output

[1306] Art data stored in the buffer, initial sensor settings

[1307] Specific actions

[1308] The server sends the generated interactive art data to the device using a REST API. The device stores the received data in a buffer, prepares for projection, and checks and initializes the sensor settings.

[1309] python

[1310] response = requests.post('http: / / endpoint_to_terminal', json=art_data)

[1311] with open('art_data_buffer.bin', 'wb') as buffer_file:

[1312] buffer_file.write(response.content)

[1313] Step 4: Detecting user movements and processing data

[1314] input

[1315] User movement data detected by sensors (e.g., distance, type of movement)

[1316] output

[1317] Dynamically adjusted art data

[1318] Specific actions

[1319] As users move through public spaces, sensors detect their movements in real time, measuring the distance and type of movement of passersby and transmitting that data to the device, which then dynamically adjusts and projects the generated art.

[1320] python

[1321] sensor_data = sensor.read_data()

[1322] if sensor_data['distance'] < threshold:

[1323] update_art_display('interaction_type', 'color_change')

[1324] Step 5: Collect and fuse local data

[1325] input

[1326] Regional historical elements and character data

[1327] output

[1328] New prompts and design data provided to the generative AI

[1329] Specific actions

[1330] The server collects data on local historical elements and popular characters, and instructs the AI ​​to analyze it. The AI ​​then combines this data into interactive art to generate new art data, which can be used in local events and promotional activities.

[1331] sql

[1332] SELECT FROM regional_data WHERE region = 'Shibuya';

[1333] Prompt Sentence Examples

[1334] "Create interactive art that includes historical elements and famous characters from Shibuya."

[1335] This will enable interactive art exhibitions, allowing the art to change in real time in response to the movements of visitors, and will also strengthen ties with the local community.

[1336] (Application example 1)

[1337] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1338] In modern public facilities and commercial areas, static decorations and advertisements do not have enough impact on passersby, making it difficult to increase customer engagement. It is also difficult to bring out the unique charm of a region without utilizing its historical elements and character. Furthermore, it is necessary to provide a more engaging experience by incorporating interactive elements that recognize passersby's movements and behavior in real time and dynamically change accordingly. A new system is needed to solve these issues, increase customer engagement, and improve the appeal of commercial areas and public facilities.

[1339] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1340] In this invention, the server includes means for acquiring designs drawn by artists, means for analyzing the acquired designs and generating interactive elements that change in response to the movements of passersby, terminal means for receiving and preparing the generated interactive art data, sensor means for detecting the movements of passersby, projection means for dynamically adjusting the art data based on the data from the sensors and projecting it in a public space, collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation means, implementation means for projecting the generated interactive art as a decorative display inside the store, and means for increasing customer engagement through the interactive art. This makes it possible to provide a more attractive and interactive experience and increase customer engagement compared to static decorations or advertisements.

[1341] An "artist-drawn design" is a work of art created by an artist, expressed as a visual element and including tags and metadata.

[1342] "Means for obtaining" refers to the methods and technical devices for obtaining the artist-drawn designs from the database.

[1343] "Analysis and generation of interactive elements that change in response to the movements of passersby" refers to the method and technical device for analyzing an artist's design using generative AI and generating interactive art that includes dynamic elements that change in response to the movements of passersby.

[1344] "Terminal means for receiving and preparing generated interactive art data" refers to a terminal device that receives interactive art data sent from the server and prepares it for projection.

[1345] "Sensor means for detecting the movement of passersby" refers to various sensors and their systems for detecting the distance and movement of passersby in real time.

[1346] "Projection means for dynamically adjusting art data based on data from sensors and projecting it into a public space" refers to a method and technical device for updating interactive art data based on data acquired by sensors and using a projector to project art into a public space.

[1347] "Collaboration means for collecting data on regional historical elements, characters, etc. and providing it to the generation means" refers to the methods and technical devices for collecting historical aspects and character information related to a specific region and providing it to the generation AI.

[1348] "Interior store decorative displays" refer to devices that are installed on walls, floors, etc. inside commercial facilities to display designs and information.

[1349] "Means to increase customer engagement" refers to methods and technological devices that increase customer relationships and interest through interactive experiences and art.

[1350] This invention is a system that enhances customer engagement in brick-and-mortar stores through interactive art based on artist designs. The system is primarily comprised of a server, terminals, and users, each of which plays a role in generating and projecting interactive art.

[1351] Server Roles

[1352] The server first retrieves the artist's design from the database. This design is visually represented and includes tags and metadata as additional information. The server then passes the retrieved design to a generation AI, which analyzes each element in the design and generates interactive art. The generated interactive art data is then sent from the server to the device.

[1353] Device Role

[1354] The device receives the interactive art data sent from the server and stores it in a buffer. The device prepares for projection and also checks and initializes the settings to receive sensor data in real time. This ensures that the device is always ready to project the latest interactive art.

[1355] User Roles

[1356] As users (customers) move around the store, sensors detect their movements. The sensors accurately measure the user's distance and actions (e.g., waving, jumping, etc.). The device that receives this data dynamically adjusts the generated art data and projects it onto the walls and floors inside the store.

[1357] Hardware and Software Details

[1358] Hardware

[1359] Cameras and sensors: to detect user movement and distance

[1360] Projector: for projecting interactive art

[1361] software

[1362] OpenCV: for image processing and display

[1363] TensorFlow / Keras: for using generative AI models

[1364] The server retrieves the artist's design and uses generative AI to analyze and generate interactive art. The generated art data is adjusted in real time based on customer movements detected by sensors and projected inside the store via a projector. This provides a more engaging and interactive experience than static decorations or advertisements, and can improve customer engagement.

[1365] Specific examples and prompts for the generative AI model

[1366] Specific examples

[1367] Art projected onto the walls of the shopping mall changes into a flower design when passersby approach, and it also recognizes certain actions (e.g. jumping) and displays special effects according to the action.

[1368] Example prompts

[1369] Artist Design: Cityscape

[1370] Effect: The city changes from day to night as passersby approach

[1371] Interaction: Buildings change color when passersby wave

[1372] Regional elements: Designs incorporating regional history and famous characters

[1373] This example shows how store décor can go beyond being just a visual element to provide customers with a dynamic and interactive experience, capturing their attention, increasing their dwell time in the store, and increasing their engagement.

[1374] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1375] Step 1:

[1376] The server retrieves the artist's drawing from the database.

[1377] Input: Artist's design ID

[1378] Data processing: Search the database based on the design ID and obtain the corresponding design data

[1379] Output: Artist's design (image file and additional information)

[1380] Step 2:

[1381] The server passes the acquired design to the generative AI model and begins analysis.

[1382] Input: Artist's design (image file and additional information)

[1383] Data Computation: Generative AI models analyze the visual elements of a design and generate dynamic, interactive elements.

[1384] Output: Interactive art data

[1385] Step 3:

[1386] The server transmits the generated interactive art data to the terminal.

[1387] Input: Interactive art data

[1388] Data processing: Converting interactive art data into a format that can be received by the device

[1389] Output: Converted interactive art data

[1390] Step 4:

[1391] The terminal stores the received interactive art data in a buffer and prepares it for projection.

[1392] Input: Generated interactive art data

[1393] Data processing: Save interactive art data to a buffer, check and initialize projector settings

[1394] Output: The projector is ready to project.

[1395] Step 5:

[1396] As users (customers) move around the store, sensors detect their movements.

[1397] Input: User movement (distance and action)

[1398] Data processing: Motion detected by sensors is converted into data in real time

[1399] Output: Detected motion data

[1400] Step 6:

[1401] The device dynamically adjusts the art data based on data from the sensors and performs the projection.

[1402] Input: User movement data from sensors, interactive art data

[1403] Data Calculation: Dynamically update the color and shape of interactive art based on user movement

[1404] Output: Updated interactive art (projection)

[1405] Step 7:

[1406] The server collects data on the region's historical elements, characters, etc. and provides it to the generative AI model.

[1407] Input: Regional historical data, character data

[1408] Data processing: Converting collected data into a format that can be analyzed by the AI ​​model

[1409] Output: Interactive art data including regional data

[1410] Step 8:

[1411] The terminal then projects the final interactive art as a decorative display inside the store.

[1412] Input: Final generated interactive art data

[1413] Data processing: Projecting interactive art data optimized for display

[1414] Output: Interactive art projected onto decorative displays inside the store

[1415] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1416] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[1417] System Overview

[1418] 1. The server acquires and analyzes the artist's design

[1419] The first step the server takes is to retrieve the artist's design from the database. The retrieved design includes the artist's intended visual expression as well as tags and metadata for adding interactive elements. The server then sends the retrieved design data to the generation AI, instructing it to generate interactive elements that change in response to the movements and emotions of passersby.

[1420] 2. Generating Interactive Art with Generative AI

[1421] The generative AI analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change according to the movements and emotions of passersby. This data is then sent to the device via the server.

[1422] 3. Data reception by the terminal and preparation for projection

[1423] The device receives the generated interactive art data from the server, stores this data in a buffer, and initializes the device to display the art in real time. Additionally, the device prepares to receive data from the sensors and emotion engine in real time.

[1424] 4. User movement detection and emotion recognition

[1425] When a user (passerby) passes through a public space, a sensor detects their movement. The data measured by the sensor is sent to the device in real time. At the same time, the emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[1426] 5. Device processing of sensor and emotion data

[1427] The device receives and analyzes movement data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, the color and shape of the art may change when the user approaches, or the color may become brighter if the user smiles.

[1428] 6. Projecting art using projection media

[1429] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[1430] 7. Collaboration that combines local elements

[1431] The server collects data on historical elements and popular characters from the region, and then instructs the AI ​​to perform further analysis based on that data. The AI ​​then combines these elements into the art, generating new interactive art. This new art data is then sent back to the device via the server, and can be used in local promotional events and other events.

[1432] Specific examples

[1433] As a specific example, consider a scenario in which new art is projected onto a famous street in Tokyo, such as Shibuya.

[1434] Server: Initial setup and art design acquisition

[1435] The server will acquire a Shibuya-themed design by an artist, which will include iconic Shibuya scenery and landmarks.

[1436] Generative AI: Generating Interactive Art

[1437] The AI ​​analyzes the Shibuya scenery and generates art that changes the time of day from night to day as passersby approach. This interactive art changes dynamically according to the movements and emotions of passersby.

[1438] Terminal: Receiving art data and preparing for projection

[1439] The generated art is sent to the device, which prepares it for projection onto the wall. The device monitors passersby's movements via sensors and their emotions via an emotion engine in real time.

[1440] User (passerby): Motion and emotion detection, and art changes

[1441] Sensors detect the movements of passersby, and an emotion engine analyzes their facial expressions to identify their emotions. This data is then sent to the device. For example, if a passerby approaches the wall and smiles, the colors of the artwork will become more vibrant and the scenery will change to reflect their emotions.

[1442] Collaboration with the local community

[1443] The server collects data related to Shibuya's history and culture and instructs the AI ​​to incorporate it. The new art generated will incorporate Shibuya's traditional festivals and famous characters and will be unveiled at local promotional events.

[1444] This system provides passersby with a deeper art experience, drawing out the local area's charm and promoting its revitalization. In addition, by combining it with an emotion engine, it becomes possible to provide individual experiences that correspond to the user's emotions.

[1445] The processing flow will be explained below.

[1446] Step 1:

[1447] The server retrieves the artist's design from the database. Specifically, it searches for the design data using the specified artist ID or project ID and downloads the necessary files.

[1448] Step 2:

[1449] The server sends the acquired design data, which includes tags and metadata for adding interactive elements, to the generation AI, which analyzes the design data and generates interactive elements that change in response to the movements and emotions of passersby.

[1450] Step 3:

[1451] The generation AI returns the generated interactive art data to the server, which then prepares it for transmission to the device.

[1452] Step 4:

[1453] The device receives the generated interactive art data from the server. The device stores this data in a buffer and performs initial setup to display the art in real time. It also prepares settings to receive data from the emotion engine and sensors.

[1454] Step 5:

[1455] When a user (passerby) passes through a public space, sensors detect the user's movements, for example, by measuring the user's distance and type of action (e.g., waving, jumping, etc.).

[1456] Step 6:

[1457] The emotion engine analyzes the user's facial expressions and recognizes their emotions. Specifically, it uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger.

[1458] Step 7:

[1459] The sensor and emotion engine send measured data to the device in real time, and the device analyzes the received sensor data and emotion data to identify the user's actions and emotions.

[1460] Step 8:

[1461] The device dynamically readjusts the generated interactive art based on data from sensors and an emotion engine, for example, changing the color of the art when the user approaches, or turning brighter if the user smiles.

[1462] Step 9:

[1463] The terminal transmits the adjusted interactive art data to a projection device, which projects it onto a public space such as a wall. The projector displays the art in high resolution and waits for the next sensor and emotion data.

[1464] Step 10:

[1465] The server collects data on local historical elements and popular characters, including images, descriptions, and 3D models.

[1466] Step 11:

[1467] The server sends the collected data to the AI ​​generator, which then instructs it to incorporate these elements into the art, generating new interactive art that incorporates local elements.

[1468] Step 12:

[1469] The server sends the newly generated art data to the terminal, which again receives the art data and updates the projection system, preparing to project the new interactive art into the public space.

[1470] Step 13:

[1471] The terminals will project new interactive art that will be used for local events and promotional activities, helping to revitalize the area.

[1472] Example 2

[1473] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1474] In systems that project interactive art in public spaces, there is a need to provide a richer art experience by reflecting not only the movements but also the emotions of passersby. Another important issue is to contribute to the revitalization of the local area by incorporating local history and culture.

[1475] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation AI means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means for detecting the movements of passersby; emotion recognition means including an emotion engine that recognizes the emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion recognition means and projecting it in a public place; and collaboration means for collecting data on local historical elements, characters, etc. and providing it to the generation AI means. This makes it possible to generate interactive art that changes color and shape based on the movements and emotions of passersby and is used in local events and promotional activities.

[1476] "Artist-drawn design" means design data created by an artist that includes visual expressions and interactive elements.

[1477] "Generative AI means" is an artificial intelligence technology that analyzes the artist's design and generates interactive elements that change in response to the movements and emotions of passersby.

[1478] "Terminal means" refers to a device that receives interactive art data generated from the server and prepares it for real-time display and projection.

[1479] "Sensor means" refers to equipment or technology used to detect the movement of passersby.

[1480] The "emotion recognition means" is an emotion engine that includes face recognition technology to recognize the emotions of passersby.

[1481] "Projection means" means a device for projecting adjusted art data into a public space.

[1482] "Collaboration means" refers to mechanisms or means for collecting data on local historical elements, characters, etc., and providing it to the generative AI means.

[1483] This invention combines a system in which a generative AI projects interactive art in public spaces based on designs drawn by artists, with an emotion engine that recognizes the user's emotions. The main components are a server, terminals, sensors, the emotion engine, and the user.

[1484] The server retrieves the artist's design from the database. The retrieved design includes visual and interactive elements. The server sends this design data to the generation AI, instructing it to generate interactive elements that change according to the movements and emotions of passersby. For example, the server retrieves the design data from a MySQL database and sends a POST request to the generation AI's API.

[1485] The generative AI then analyzes the design data received from the server and generates interactive art. The generated art data incorporates elements that change in response to the movements and emotions of passersby. For example, it includes instructions such as "change color when a user approaches." This data is sent to the device via the server.

[1486] The device receives the generated interactive art data from the server and stores it in a buffer. The device prepares to receive data from sensors and emotion engines in real time. For example, the device receives art data from an AWS EC2 server and stores it in local storage.

[1487] When a user (passerby) passes through a public space, a sensor detects their movement. The data captured by the sensor is sent to the device in real time. At the same time, an emotion engine analyzes the user's facial expressions and recognizes their emotions. The emotion engine uses facial recognition technology to evaluate the user's facial expressions in real time and identify emotions such as joy, surprise, sadness, and anger. For example, Microsoft Kinect is used as the sensor and Affectiva SDK is used as the emotion engine.

[1488] The device receives and analyzes motion data from sensors and emotion data from the emotion engine. Based on this, it dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. This adjusted art data is then sent to the projection device.

[1489] The projection device projects the adjusted interactive art data in high resolution into a public space. The projector continues to display the art while waiting for the next sensor and emotion data. For example, an EPSON projector is used to project the art.

[1490] Another feature of this system is its collaboration with local elements. The server collects data on the region's history, culture, popular characters, and other topics, and sends it to the AI ​​generator. The AI ​​analyzes this data and generates new interactive art. This art data is then sent back to the device via the server and used in local promotional events.

[1491] An example of a prompt is, "Using design data themed on the city of Shibuya, generate interactive art in which the scenery changes from night to day when passersby approach, and the colors change vividly when passersby smile." By inputting this prompt into a generative AI model, specific art is generated.

[1492] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1493] Step 1:

[1494] The server retrieves the artist's design. Specifically, the server connects to a database (e.g., MySQL) and executes the query "SELECT FROM designs WHERE artist_id = '12345'". The input is the database connection information and the query, and the output is the retrieved design data. This design data includes visual and interactive elements.

[1495] Step 2:

[1496] The server sends the acquired design data to the generative AI. Specifically, the server sends an HTTP POST request to the API endpoint of the generative AI model, including the query result design data in JSON format. The input is the acquired design data, and the output is interactive art data generated by the generative AI model.

[1497] Step 3:

[1498] The generative AI analyzes the design data received from the server and generates interactive art. In this process, the generative AI analyzes the visual and interactive elements contained in the design data and generates art that changes in response to movement and emotions. The input is the design data sent from the server, and the output is the generated interactive art data.

[1499] Step 4:

[1500] The device receives the generated interactive art data from the server and stores it in a buffer. Specifically, the device retrieves the data from the server via an HTTP GET request and stores it in local storage (e.g., / tmp / art_data.json). The input is the interactive art data returned by the generation AI, and the output is the stored art data.

[1501] Step 5:

[1502] When a user (passerby) passes through a public space, a sensor detects their movement. Specifically, the sensor (e.g., Microsoft Kinect) captures the user's movement and transmits the data to a device in real time. The input is the user's movement, and the output is the detected movement data.

[1503] Step 6:

[1504] The emotion engine analyzes the user's facial expressions and recognizes emotions. Specifically, the emotion engine (e.g., Affectiva SDK) captures the user's face in real time and generates emotion data (happiness, surprise, sadness, anger, etc.). The input is the user's facial expression, and the output is the recognized emotion data.

[1505] Step 7:

[1506] The device receives and analyzes sensor movement data and emotion data from the emotion engine. Specifically, the device analyzes the data received in real time and dynamically adjusts the interactive art. For example, it brightens the color tone when the user smiles, and changes the image composition when the user approaches. The input is data from the sensor and emotion engine, and the output is the adjusted art data.

[1507] Step 8:

[1508] The terminal transmits the adjusted art data to a projection device, which projects it into a public space. Specifically, the terminal transmits the art data to a projector (e.g., an EPSON projector) and projects it in high resolution. The input is the adjusted interactive art data, and the output is the art projected into a public space.

[1509] Step 9:

[1510] The server collects regional data (history, characters, etc.) and sends it to the generation AI. Specifically, the server collects regional data from APIs and databases and sends it to the generation AI. The input is regional data, and the output is the generated interactive art specific to the region.

[1511] Step 10:

[1512] Interactive art specific to the region is generated and sent to the device. Specifically, the generation AI analyzes the region data received from the server, generates new interactive art, and sends it to the device. The input is the region data from the server, and the output is the generated interactive art data specific to the region.

[1513] (Application example 2)

[1514] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1515] Currently, public spaces and commercial facilities mainly present static advertisements and art exhibits to passersby and customers, lacking in interactive experiences. Furthermore, there has been little progress in practical application of technology that provides dynamic content that reflects the emotions of passersby and customers. This means that traditional advertisements and exhibits are one-way, making it difficult to attract users' attention. Furthermore, there is a lack of content that reflects local culture and history, making it difficult to contribute to regional revitalization.

[1516] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a design drawn by an artist; generation means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby; terminal means for receiving and preparing the generated interactive art data; sensor means and emotion engine means for detecting the movements and emotions of passersby; projection means for dynamically adjusting the art data based on data from the sensor and emotion engine and projecting it in a public place or commercial facility; and collaboration means for collecting local cultural elements and providing them to the generation means. This makes it possible to provide passersby and customers with an interactive, dynamic art experience that responds to their emotions, and to contribute to regional revitalization with content that reflects the local culture and history.

[1517] "Means for capturing artist-drawn designs" refers to any equipment or software used to receive or capture in digital form the visual designs created by the artist.

[1518] "Generation means" refers to software or algorithms that generate interactive elements based on acquired design data.

[1519] "Terminal means" refers to a process or device for receiving, displaying, and projecting the generated art data.

[1520] "Sensor means" refers to devices or technologies for detecting the movement of users or passersby.

[1521] "Emotion engine means" refers to software or algorithms that analyze the facial expressions of users and passersby and identify their emotions.

[1522] "Projection means" refers to a projector or related device for projecting art data onto a wall or other surface in a public place or commercial facility.

[1523] "Collaboration tools" refer to databases and software systems that collect local cultural elements and provide them to production tools.

[1524] MODE FOR CARRYING OUT THE INVENTION

[1525] Overall system configuration

[1526] This invention is a system for projecting interactive art in public places and commercial facilities based on designs drawn by artists. This system is composed of a generation means for analyzing and generating acquired design data, a sensor means and emotion engine means for detecting movement and emotions, a terminal means for receiving and displaying data, a projection means for projecting the art, and a collaboration means for collecting local cultural elements.

[1527] Specific examples of each component

[1528] server

[1529] The server retrieves and analyzes the artist's design. The design data includes tags and metadata for adding interactive elements. The server then sends this design data to the AI, instructing it to generate interactive art based on the movements and emotions of passersby.

[1530] generation means

[1531] The generation method uses a generative AI model (e.g., OpenAI's DALL-E or GPT-4), analyzes the design data received from the server, and generates interactive art that changes in response to the user's movements and emotions.

[1532] Terminal means

[1533] The terminal means receives the generated interactive art data, stores the data in a buffer, performs initialization for displaying the art in real time, and prepares to receive data from the sensor means and emotion engine means in real time.

[1534] Sensor Means and Emotion Engine Means

[1535] The sensor means uses cameras and IR sensors, which detect the movements of users (passersby) in real time. The emotion engine means uses Microsoft Azure Cognitive Services API to analyze facial expressions and identify emotions such as joy, surprise, sadness, and anger.

[1536] Projection Method

[1537] A high-resolution projector is used as a projection medium, and the generated interactive art data is projected into public spaces or commercial facilities.

[1538] Collaboration tools

[1539] The server collects local cultural elements and provides them to the generation means, allowing the generation AI to generate interactive art that incorporates new elements.

[1540] Example

[1541] Specific examples include the following:

[1542] Interactive art in public spaces

[1543] The server retrieves designs that reflect the characteristics of the area, and the AI ​​generates interactive art that changes in response to the movements and emotions of passersby.

[1544] Example prompt sentence:

[1545] Generate an art piece that changes color and form based on user emotions and movements detected via sensors.

[1546] The device receives data from the sensors and configures it to display art in real time.

[1547] When passersby smile, the projected art changes into vibrant colors and adds animation effects.

[1548] Art exhibition in a commercial facility

[1549] The server retrieves designs based on the theme of the commercial facility and generates interactive art using generative AI.

[1550] The device displays art on its display and receives data from sensors and an emotion engine.

[1551] As customers pass by the display in the store, the art changes based on their movements and facial expressions, providing a more enjoyable shopping experience.

[1552] In this way, the system implementing the invention can effectively combine an improved art experience with local cultural elements.

[1553] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1554] Step 1:

[1555] The server retrieves the artist's design from the database. When retrieving the design data, it also includes tags and metadata for adding interactive elements. This becomes the input data for the server. The retrieved design data is then analyzed in digital format and processed into a format that can be sent to the generation AI. This processed data becomes the output data from the server to the generation AI.

[1556] Step 2:

[1557] The server sends the acquired design data to the generation AI, which then generates interactive art based on a prompt (e.g., "Generate an art piece that changes color and form based on user emotions and movements detected via sensors."). The generation AI analyzes the data and generates art, and the generated interactive art data is the output.

[1558] Step 3:

[1559] The server receives the interactive art data generated by the generation AI and sends it to the terminal. The terminal stores the received data in a buffer and performs initial settings to display the art in real time. This becomes input data to the terminal. The terminal prepares to receive data from the sensor means and emotion engine means in real time.

[1560] Step 4:

[1561] When a user (passerby) visits a public place or commercial facility, the sensor means detects the user's movements, and at the same time the emotion engine means analyzes the user's facial expressions. The movement data and emotion data acquired by the sensor and emotion engine become input data to the terminal. Based on this data, the terminal performs data analysis and dynamically adjusts the art data.

[1562] Step 5:

[1563] The device adjusts the interactive art data in real time based on the motion and emotion data. For example, it can change the color or shape of the art when the user approaches, or change to a brighter color tone when the user smiles. This adjusted art data becomes input data for the projection means.

[1564] Step 6:

[1565] The device then transmits the adjusted interactive art data to a projector, which projects it into a public or commercial space. The projector displays the art in high resolution and waits for new sensor and emotion data input. The displayed art allows users to enjoy an interactive experience.

[1566] Step 7:

[1567] The server collects local cultural and historical elements from a database and provides this data to a generation AI, which then generates new interactive art incorporating elements unique to the region or commercial facility. This newly generated art data becomes output data for the server and can be used again on devices or projection means.

[1568] This process brings together the server, the device, and the user to create a dynamic interactive art experience.

[1569] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1570] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1571] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1572] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1573] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1574] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1575] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1576] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1577] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1578] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1579] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1580] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1581] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1582] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1583] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1584] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1585] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1586] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1587] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1588] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1589] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1590] The following is further disclosed regarding the above embodiment.

[1591] (Claim 1)

[1592] A means of obtaining the artist's design;

[1593] a generating means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby;

[1594] terminal means for receiving and preparing the generated interactive art data;

[1595] a sensor means for detecting the movement of passersby;

[1596] a projection means for dynamically adjusting art data based on data from the sensor and projecting the art data into a public space;

[1597] A collaborative method for collecting data on local historical elements and popular characters and providing it to the generation method.

[1598] A system including:

[1599] (Claim 2)

[1600] The system of claim 1, wherein the art changes color or shape based on the movement of passersby.

[1601] (Claim 3)

[1602] 10. The system of claim 1, used for local events and promotional activities.

[1603] "Example 1"

[1604] (Claim 1)

[1605] A means of obtaining the design;

[1606] A generating means for analyzing the acquired design and generating elements that change according to the movement;

[1607] terminal means for receiving and preparing the generated interactive art data;

[1608] a sensor means for detecting movement;

[1609] a projection means for dynamically adjusting the art data based on data from the sensor and projecting the art data into a public space;

[1610] A collaborative means of collecting data on local historical elements and characters and providing it to the generation means;

[1611] A system including:

[1612] (Claim 2)

[1613] 10. The system of claim 1, wherein the system changes color or shape of the art based on movement.

[1614] (Claim 3)

[1615] 10. The system of claim 1, used for local events and promotional activities.

[1616] "Application Example 1"

[1617] (Claim 1)

[1618] A means of obtaining the artist's design;

[1619] a generating means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby;

[1620] terminal means for receiving and preparing the generated interactive art data;

[1621] a sensor means for detecting the movement of passersby;

[1622] a projection means for dynamically adjusting art data based on data from the sensor and projecting the art data into a public space;

[1623] A collaborative method for collecting data on local historical elements and characters and providing it to the generation method.

[1624] an implementation means for projecting the generated interactive art as a decorative display inside the store;

[1625] Interactive art is a way to increase customer engagement,

[1626] A system including:

[1627] (Claim 2)

[1628] The system of claim 1, wherein the art changes color or shape based on the movement of passersby.

[1629] (Claim 3)

[1630] 10. The system of claim 1, used for store events and promotional activities.

[1631] "Example 2: Combining Emotion Engines"

[1632] (Claim 1)

[1633] A means of obtaining the artist's design;

[1634] A generating AI means for analyzing the acquired design and generating interactive elements that change in response to the movements and emotions of passersby;

[1635] terminal means for receiving and preparing the generated interactive art data;

[1636] a sensor means for detecting the movement of passersby;

[1637] emotion recognition means including an emotion engine for recognizing the emotions of passersby;

[1638] a projection means for dynamically adjusting the art data based on data from the sensor and the emotion recognition means and projecting the art data into a public place;

[1639] A collaborative method for collecting data on local historical elements and characters and providing it to the AI ​​generation method.

[1640] A system including:

[1641] (Claim 2)

[1642] The system of claim 1, wherein the art changes color and shape based on the movement and emotion of passersby.

[1643] (Claim 3)

[1644] 10. The system of claim 1, used for local events and promotional activities.

[1645] "Application example 2 when combining emotion engines"

[1646] (Claim 1)

[1647] A means of obtaining the artist's design;

[1648] a generating means for analyzing the acquired design and generating interactive elements that change according to the movements and emotions of passersby;

[1649] terminal means for receiving and preparing the generated interactive art data;

[1650] a sensor means and an emotion engine means for detecting the movement and emotion of a passerby;

[1651] a projection means for dynamically adjusting the art data based on data from the sensor and the emotion engine and projecting the art data within a public or commercial space;

[1652] A collaborative means of collecting and providing local cultural elements for generation;

[1653] A system including:

[1654] (Claim 2)

[1655] The system of claim 1, wherein the art changes color and shape based on the movement and emotion of passersby.

[1656] (Claim 3)

[1657] 10. The system according to claim 1, which is used for promotional activities of local events and commercial facilities. [Explanation of symbols]

[1658] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining the artist's design; a generating means for analyzing the acquired design and generating interactive elements that change in response to the movements of passersby; terminal means for receiving and preparing the generated interactive art data; a sensor means for detecting the movement of passersby; a projection means for dynamically adjusting art data based on data from the sensor and projecting the art data into a public space; A collaborative method for collecting data on local historical elements and popular characters and providing it to the generation method. A system including:

2. The system of claim 1 , wherein the color and shape of the art changes based on the movement of passersby.

3. The system of claim 1, used for local events and promotional activities.

Images