System

Abstract

A system is provided.SOLUTION: A system comprising: means for linking a user with a social networking service; means for acquiring user data from the social networking service; means for analyzing the user data and creating a profile of the user; means for generating an alter ego of the user in a virtual space based on the profile; means for collecting behavior data of the alter ego of the user in the virtual space; means for analyzing the collected behavior data; and means for visualizing an analysis result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While individual users have become more adept at creating high-quality applications in recent years, companies still face challenges in collecting and utilizing large-scale, high-quality data. It is also difficult for companies to collect large amounts of high-quality data directly from users, due to concerns about user privacy and data security. Furthermore, existing data collection methods make it difficult to accurately grasp users' detailed behavioral patterns and interests, creating a demand for more sophisticated data collection methods. The present invention aims to solve these data collection challenges and provide a system for efficiently collecting high-quality data. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides a system including: a means for users to connect with a social networking service; a means for acquiring user data from the social networking service; a means for analyzing the user data and creating a user profile; a means for generating a user avatar in a virtual space based on the profile; a means for collecting behavioral data of the user avatar in the virtual space; a means for analyzing the collected behavioral data; and a means for visualizing the analysis results. Furthermore, by including a means for analyzing user data using a natural language processing algorithm, the system accurately analyzes and extracts user interests. Furthermore, by including a means for the user avatar to interact with other avatars in the virtual space, the system enables the collection of more detailed behavioral data. This allows companies to efficiently collect and utilize high-quality, detailed data, contributing to the creation of new value and the improvement of services.

[0006] "User" means an individual or entity that uses a social networking service.

[0007] A "social networking service" is a service that enables users to interact with and share information online.

[0008] "User data" refers to data such as user posts, comments, images, and like history collected on social networking services.

[0009] A "profile" is information generated based on user data that represents a user's interests, personality, behavioral patterns, etc.

[0010] A "generative AI model" is a model that uses artificial intelligence techniques to generate new data from input data.

[0011] A "virtual space" is a digital environment constructed using virtual reality technology in which users can interact and engage in activities.

[0012] A "user's alter ego" is an avatar that is generated based on the user's profile and lives and acts within a virtual space.

[0013] "Behavioral data" refers to data relating to the behavior and interactions of the user's avatar within the virtual space.

[0014] A "natural language processing algorithm" is an algorithm that allows a computer to understand, analyze, and generate human language.

[0015] "Visualization tools" are means of displaying collected and analyzed data in visual formats such as graphs and charts.

[0016] A "tracking system" is a system that tracks the actions of a user's avatar in a virtual space in real time. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that allows users to connect with social networking services (SNS), analyzes user data obtained from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[0039] Program processing overview

[0040] 1. Data Collection

[0041] Device: The user links their SNS account to the application. At this time, the user enters their SNS account authentication information and grants access rights to the service.

[0042] Server: Calls the SNS API based on the user's authentication information and obtains data such as the user's posts, comments, images, and like history.

[0043] 2. Data Analysis

[0044] Server: Preprocesses the acquired SNS data, for example by cleaning text and normalizing images.

[0045] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data. Also performs image analysis to extract features to identify user interests.

[0046] Server: Generates a user profile based on these analysis results.

[0047] 3. User avatar generation

[0048] Server: Enters profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space.

[0049] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[0050] 4. Collection of lifestyle data

[0051] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[0052] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0053] 5. Data Analysis and Visualization

[0054] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[0055] Server: Visually displays the analysis results using visualization tools, allowing companies to easily understand and utilize the data.

[0056] Specific examples

[0057] Examples of data collection

[0058] Device: The user launches the application and clicks the button to link their Facebook account.

[0059] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[0060] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[0061] Specific examples of data analysis

[0062] Server: In order to analyze the collected data, unnecessary tags and special characters are removed from the text data.

[0063] Server: Uses natural language processing models to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[0064] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[0065] Example of user avatar creation

[0066] Server: Based on the analysis results, a profile of the user is created with characteristics such as "travel lover" or "nature lover."

[0067] Server: This profile is input into the generative AI model, and an avatar of the user is generated in the virtual space.

[0068] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[0069] Examples of collecting lifestyle data

[0070] Server: Records the actions of your avatar in real time, storing logs of the places you visit and the activities you participate in.

[0071] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail and analyzes the user's interests and behavioral patterns.

[0072] Specific examples of data utilization

[0073] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[0074] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[0075] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] Device: The user launches the application and clicks the button to link their social media account.

[0079] Step 2:

[0080] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[0081] Step 3:

[0082] Device: When the user authorizes the connection, an authentication token is generated.

[0083] Step 4:

[0084] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[0085] Step 5:

[0086] Server: Saves the acquired user data in a database.

[0087] Step 6:

[0088] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[0089] Step 7:

[0090] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[0091] Step 8:

[0092] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[0093] Step 9:

[0094] Server: Creates a user profile based on all analysis results.

[0095] Step 10:

[0096] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[0097] Step 11:

[0098] Server: Places the generated avatar at its initial position in the virtual space.

[0099] Step 12:

[0100] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[0101] Step 13:

[0102] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0103] Step 14:

[0104] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[0105] Step 15:

[0106] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[0107] Step 16:

[0108] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[0109] This allows companies to create detailed profiles of users from their social media data, generate virtual avatars based on those profiles, and track their behavior. The collected data is a very useful source of information for companies.

[0110] Example 1

[0111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] In today's information society, companies and organizations aim to create new value and improve their services by analyzing and utilizing detailed user behavior data. However, with current technology, it has been difficult to efficiently analyze and visualize detailed data, including not only users' online behavior but also their behavioral patterns in virtual environments. Furthermore, technologies for automatically generating and tracking behavior in virtual environments directly from user data have been limited, restricting companies' multifaceted use of data.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0114] In this invention, the server includes a means for a user to link with the information sharing system, a means for acquiring user data from the information sharing system, a means for analyzing the user data and creating user characteristics, a means for generating a user representative in a virtual environment based on the characteristics, a means for collecting activity data of the user representative in the virtual environment, a means for analyzing the collected activity data, and a means for visualizing the analysis results. This enables comprehensive analysis of user behavior data online and in the virtual environment, enabling companies to efficiently utilize detailed data.

[0115] "User" refers to an individual or organization that uses the system.

[0116] "Information sharing system" refers to online platforms that provide user data, such as social networking services.

[0117] "User Data" refers to digital information obtained from the information sharing system, such as user posts, comments, images, and like history.

[0118] "Analysis" refers to the process of processing acquired user data and extracting useful information and trends.

[0119] "Traits" refer to information about a user's interests and behavioral patterns generated from analyzed data.

[0120] A "virtual environment" refers to a digital space simulated by a computer.

[0121] "Proxy" refers to a digital avatar that is generated to simulate the user's actions within a virtual environment.

[0122] "Activity Data" refers to a record of actions taken by an agent within a virtual environment.

[0123] "Visualization" refers to the process of displaying analytical results in the form of diagrams, charts, etc., to make the data easier to understand.

[0124] MODE FOR CARRYING OUT THE INVENTION

[0125] The present invention is a system that allows users to connect to an information sharing system, analyzes the user data acquired from the system, creates user characteristics, and generates a user's agent in a virtual environment. It also provides a platform that collects and analyzes the agent's activity data in the virtual environment and enables companies to utilize detailed data.

[0126] 1. Data collection and collaboration

[0127] Terminal: To access the information sharing system, a user launches an application on the terminal and clicks, for example, a "Log in with Facebook" button. The user enters their login information and grants the necessary data access permissions to link with the information sharing system.

[0128] Server: Using the obtained authentication token, the server calls the API of the information sharing system and obtains data such as the user's posts, comments, images, and like history. For example, the server obtains user data using the Facebook API.

[0129] 2. Data Analysis

[0130] Server: Preprocesses the collected data. For example, it uses Python regular expressions to remove unnecessary HTML tags and special characters from text data. Similarly, it standardizes the format and size of image data.

[0131] Server: Uses natural language processing algorithms to extract keywords, emotions, and tones from text data. For example, it uses Python's NLTK library to classify positive and negative emotions. It also performs image analysis using OpenCV to recognize objects and scenes in images.

[0132] Server: Based on the results of these analyses, the server generates user characteristics, including user interests and behavioral patterns.

[0133] 3. Creating a User Agent

[0134] Server: Input user characteristics into the generative AI model and generate a representative with the user's characteristics in the virtual environment. For example, input the prompt "Generate a user who likes to travel in the virtual space" into the generative AI model (e.g., GPT-3).

[0135] Example prompt:

[0136] Create a virtual travel-loving user who is a nature lover and travels frequently.

[0137] Server: Places the generated agent at an initial position in the virtual environment and allows the agent to move freely. Using the virtual environment engine, imports a 3D model and performs initial settings.

[0138] 4. Agent Activity Data Collection

[0139] Server: Activates the tracking system and tracks the agent's actions in real time, for example, recording the time, location, and interactions of the agent with other users in the virtual park.

[0140] Server: The agent's movements and activities are stored in a database as log data.

[0141] 5. Data Analysis and Visualization

[0142] Server: Analyzes the collected behavioral data to identify user behavioral patterns and preferences. Classifies the agent's behavior using clustering techniques and visually analyzes the data.

[0143] Server: Displays the analysis results using visualization tools (e.g., Tableau or Power BI), allowing companies to easily understand and utilize the data.

[0144] This invention enables comprehensive analysis of users' online and virtual behavioral data, enabling companies to efficiently utilize detailed data. This system contributes to the creation of new value and the improvement of services.

[0145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0146] Step 1: Data collection

[0147] Input: Credentials for users to log in to the information sharing system

[0148] What happens: A user launches the application and clicks the "Log in with Facebook" button. The user enters their login information in the OAuth authentication screen that appears and grants the application the necessary data access permissions.

[0149] Processing: An OAuth token is obtained based on the authentication information, and the server uses the information sharing system's API to obtain data such as the user's posts, comments, images, and like history.

[0150] Output: Retrieved user data

[0151] Step 2: Data Preprocessing

[0152] Input: Acquired user data (text, images, etc.)

[0153] Specific operations: The server removes unnecessary HTML tags and special characters from text data, performs preprocessing such as using Python regular expressions, and standardizes the format and size of image data.

[0154] Processing: Clean and normalize the data.

[0155] Output: Preprocessed user data

[0156] Step 3: Data analysis

[0157] Input: Preprocessed user data (text, images, etc.)

[0158] How it works: The server uses natural language processing algorithms (e.g., Python's NLTK library) to extract keywords, emotions, and tones from text data. The server uses image analysis techniques (e.g., OpenCV) to recognize objects and scenes in images.

[0159] Processing: Extract keywords and emotions from text data, and recognize objects and scenes from image data.

[0160] Output: User characteristics as analysis results

[0161] Step 4: Creating a User Agent

[0162] Input: User characteristics

[0163] Specific operation: The server creates a prompt for the generated AI model (e.g., GPT-3) to input the user's characteristic information and inputs it into the model. The specific prompt is, "Please generate a user who loves to travel in the virtual space. This user is a nature lover and travels frequently."

[0164] Processing: A generative AI model generates a user representative in a virtual space based on input prompts.

[0165] Output: A user representative generated in the virtual space

[0166] Step 5: Collecting data on agent activities

[0167] Input: A user's representative generated in the virtual space

[0168] Specific operation: The server starts a tracking system to track the agent's actions in real time, recording the agent's location, movement route, and interactions with other agents.

[0169] Processing: Collecting agent behavior data and storing it in a database.

[0170] Output: Detailed agent behavior data

[0171] Step 6: Data analysis and visualization

[0172] Input: Detailed agent behavior data

[0173] Specific operations: Analyze the behavioral data collected by the server to identify behavioral patterns and preferences. Classify the behavioral data using clustering techniques. Display the analysis results using a visualization tool (e.g., Tableau or Power BI).

[0174] Processing: Analysis and visualization of behavioral data

[0175] Output: Analyzed behavioral patterns and preferences, visualized data

[0176] In this way, by having the user and server perform specific actions at each step, the system can perform detailed analysis of user behavior data and provide data that companies can use to create new value.

[0177] (Application example 1)

[0178] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0179] Conventional virtual user avatar systems have the problem that they do not adequately provide personalized product recommendations based on the user's interests. In addition, the collection and analysis of user behavior data in virtual space is insufficient, making it difficult for companies to obtain information that is useful for marketing strategies and new product development.

[0180] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0181] In this invention, the server includes means for allowing a user to link with a social networking service, means for acquiring user data from the social networking service, means for analyzing the user data and creating a user profile, means for generating an avatar of the user in a virtual space based on the profile, means for collecting behavioral data of the user's avatar in the virtual space, means for analyzing the collected behavioral data, means for visualizing the analysis results, and means for making product suggestions to the user in a virtual store based on the user's profile. This enables personalized product suggestions based on the user's interests and behavioral patterns, and allows companies to obtain detailed data for efficient and effective marketing strategies and new product development.

[0182] A "social networking service" is an online platform that enables users to connect, interact, and share information with other users over the Internet.

[0183] "User Data" refers collectively to information generated, shared, or engaged with by users on social networking services, including posts, comments, images, and like history.

[0184] A "virtual space" is an artificial three-dimensional space generated by a computer, in which users can interact through avatars.

[0185] An "alter ego" is an avatar generated in a virtual space based on a user's profile, and is a character that acts in the virtual space reflecting the user's actions, interests, and concerns.

[0186] A "virtual store" is a virtual store operated within a virtual space, where users can view and purchase virtual products.

[0187] "Personalized product suggestions" refers to the suggestion of individually customized products and services based on each user's interests and behavioral patterns.

[0188] "Natural language processing algorithms" are computational techniques for understanding, analyzing, and generating human language, and are used to extract meaning and sentiment from text data.

[0189] "Visualization" refers to the visual representation of data, making the results of data analysis intuitively understandable through dashboards, graphs, charts, etc.

[0190] This invention is a system that allows users to link with social networking services (SNS), analyzes user data acquired from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[0191] This system is realized with the following configuration.

[0192] Data collection

[0193] The user connects to their SNS account using their device. They enter their SNS account authentication information and grant access to the service. The server then calls the SNS API based on the user's authentication information to retrieve data such as the user's posts, comments, images, and like history.

[0194] Data analysis

[0195] The server preprocesses the acquired social media data, for example by cleaning the text and normalizing the images. It then uses natural language processing algorithms to extract keywords, emotions, and tones from the text data, and image analysis techniques to recognize objects and scenes in the images. Based on these analysis results, the server generates a user profile.

[0196] User avatar generation

[0197] The server inputs the user's profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space. The generated avatar is placed in an initial position in the virtual space and can move freely.

[0198] Collection of lifestyle data

[0199] The server starts a tracking system and tracks the actions of the avatar in real time, recording the avatar's movements and activities as log data and saving it in a database.

[0200] Data analysis and visualization

[0201] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, making it easy for companies to understand and utilize the data.

[0202] Product proposals in virtual stores

[0203] In a virtual store in a virtual space, a user's avatar interacts with other avatars, and personalized product suggestions are made based on their behavioral data, allowing for product suggestions based on the user's interests and behavioral patterns.

[0204] The main hardware required to realize this system is a smartphone, and the following software is used:

[0205] Platform: Android / iOS

[0206] Language: Java (Android), Swift (iOS)

[0207] API: Social networking service API (e.g. Facebook Graph API)

[0208] Libraries: Gson (JSON parsing), OkHttp3 (HTTP communication)

[0209] Generative AI model: Used to generate avatars based on user profiles

[0210] As a concrete example, the following prompt sentence can be used:

[0211] Example prompt sentence:

[0212] Generate an active user avatar in a virtual space based on the user's social media data (interests: travel, activity: hiking). Collect simulation data of the avatar searching for travel-related goods and activities in a virtual store. Display this data on a visualization dashboard, allowing travel-related companies to use it to propose new products.

[0213] This will enable companies to gain a detailed understanding of users' preferences and behavioral patterns, enabling them to propose more personalized products.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Step 1:

[0216] A user connects to a social networking account using a device. The user launches the application, enters their social networking account authentication information, and grants access to the service. At this time, the device sends the authentication information to the server and receives an authentication token to call the SNS API. The output is an authentication token.

[0217] Step 2:

[0218] The server uses the authentication token to call the SNS API and obtain data such as the user's posts, comments, images, and like history. This data is obtained in JSON format and temporarily stored on the server. The input is the authentication token, and the output is the user's SNS data.

[0219] Step 3:

[0220] The server preprocesses the acquired social media data. Specifically, it cleans the text (removes unnecessary tags and special characters) and normalizes the images. It uses natural language processing algorithms to extract keywords, emotions, and tones from the text data. It also uses image analysis techniques to recognize objects and scenes in the images. The input is the social media data, and the output is the analysis results.

[0221] Step 4:

[0222] The server generates a user profile based on the analysis results. This profile contains detailed information about the user, such as their interests and behavioral patterns. The profile is stored in a database. The input is the analysis results, and the output is the user profile.

[0223] Step 5:

[0224] The server inputs the user's profile information into the generative AI model and generates the user's avatar in the virtual space. The generated avatar is placed at an initial position in the virtual space and set up so that it can move freely. The input is the user profile, and the output is the user's avatar.

[0225] Step 6:

[0226] The server starts the tracking system and tracks the avatar's actions in real time. The avatar's movements and activities are recorded as log data and stored in a database. The input is the user's avatar, and the output is the behavior log.

[0227] Step 7:

[0228] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, allowing companies to easily understand and utilize the data. The input is the behavior log, and the output is the visualized analysis results.

[0229] Step 8:

[0230] In a virtual store in a virtual space, the user's avatar interacts with other avatars. The server uses this behavioral data to make personalized product suggestions. It identifies, highlights, and suggests products and services that the user may be interested in. The input is the behavioral log and user profile, and the output is product suggestions.

[0231] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0232] This invention is a system that allows users to link with social networking services (SNS), analyzes SNS data to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, by combining it with an emotion engine, we provide a platform that recognizes and analyzes the user's emotions and reflects that emotional information in the profile and behavioral data.

[0233] Program processing overview

[0234] 1. Data Collection

[0235] On the device: The user launches the application and clicks the button to link their SNS account. The user enters their login information for the SNS account and grants the app permission to access their data.

[0236] Server: Sends a request to the SNS API using the authentication token to retrieve user data (posts, comments, images, likes, etc.).

[0237] 2. Data Analysis

[0238] Server: Preprocesses the collected SNS data, cleaning text data and normalizing image data.

[0239] Server: Uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from text data. It also performs image analysis to extract features to identify user interests.

[0240] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[0241] 3. User avatar generation

[0242] Server: Enters profile information into the generative AI model and generates an avatar in the virtual space that reflects the user's characteristics and emotional state.

[0243] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[0244] 4. Collection of lifestyle data

[0245] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[0246] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0247] 5. Data Analysis and Visualization

[0248] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns and preferences, as well as emotional information.

[0249] Server: Visually displays the analysis results using visualization tools, making it easy for companies to understand and utilize the data.

[0250] Specific examples

[0251] Examples of data collection

[0252] Device: The user launches the application and clicks the button to link their Facebook account.

[0253] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[0254] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[0255] Specific examples of data analysis

[0256] Server: Preprocesses the acquired data and removes unnecessary tags and special characters from the text data.

[0257] Server: Uses natural language processing models and sentiment engines to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[0258] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[0259] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[0260] Example of user avatar creation

[0261] Server: Based on the analysis results, a user profile is created with characteristics and emotional information such as "travel lover" or "nature lover."

[0262] Server: This profile is input into a generative AI model to generate an avatar of the user in a virtual space. The avatar can then behave in a way that reflects the user's emotional state.

[0263] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[0264] Examples of collecting lifestyle data

[0265] Server: Records the actions of the avatar in real time, storing logs of the places visited and the activities participated in. The actions of the avatar also include emotional information.

[0266] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail, analyzes the user's interests and behavioral patterns, and also records emotional information.

[0267] Specific examples of data utilization

[0268] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[0269] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[0270] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services. The addition of an emotion engine makes it possible to collect detailed data that reflects user emotions, resulting in more accurate analysis results.

[0271] The processing flow will be explained below.

[0272] Step 1:

[0273] Device: The user launches the application and clicks the button to link their social media account.

[0274] Step 2:

[0275] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[0276] Step 3:

[0277] Device: When the user authorizes the connection, an authentication token is generated.

[0278] Step 4:

[0279] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[0280] Step 5:

[0281] Server: Saves the acquired user data in a database.

[0282] Step 6:

[0283] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[0284] Step 7:

[0285] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[0286] Step 8:

[0287] Server: Analyzes and recognizes the user's emotional information using the emotion engine.

[0288] Step 9:

[0289] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[0290] Step 10:

[0291] Server: Creates a user profile based on all analysis results, including emotional information.

[0292] Step 11:

[0293] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[0294] Step 12:

[0295] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[0296] Step 13:

[0297] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[0298] Step 14:

[0299] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0300] Step 15:

[0301] Server: Analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences, including emotional information.

[0302] Step 16:

[0303] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[0304] Step 17:

[0305] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[0306] This allows for the creation of a detailed profile from a user's social media data, the generation of an avatar based on that profile in a virtual space, and tracking of their behavior. The addition of an emotion engine makes it possible to collect and analyze detailed data that reflects user emotions, making it an extremely useful source of information for companies.

[0307] Example 2

[0308] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0309] In recent years, the number of users of social networking services has increased, leading to a diversification of the analysis and utilization of user data. However, conventional systems have had difficulty generating detailed profiles of users' emotions and behaviors, and tracking and analyzing their behavior in virtual spaces in real time. Furthermore, there are limited means for visually displaying the analysis results and for companies to effectively utilize them. Therefore, there is a need for a new system that can provide advanced analysis results, generate detailed profiles based on users' emotions and behaviors, and effectively utilize them in business strategies.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0311] In this invention, the server includes a means for a user to link with a communication network service, a means for acquiring user data from the communication network service, a means for preprocessing the acquired user data, a means for analyzing the user data and creating a user profile, a means for generating an avatar of the user in a virtual environment based on the profile, a means for collecting behavioral data of the user's avatar in the virtual environment, a means for analyzing the collected behavioral data, and a means for visualizing the analysis results. This makes it possible to generate a detailed user profile and track and analyze behavior in a virtual space, thereby providing a system that can effectively utilize the collected data.

[0312] "User" refers to any individual or corporation that uses this system.

[0313] "Communication network services" is a general term for services that allow users to exchange information via the Internet.

[0314] "User Data" refers to all information generated by users of communication network services, including posts, comments, images, like history, etc.

[0315] "Preprocessing" refers to the process of formatting and cleaning up data before analysis, and includes removing unnecessary tags and special characters, normalizing image data, etc.

[0316] "Language analysis algorithm" refers to an algorithm that uses natural language processing technology to extract useful information (keywords, emotions, tone, etc.) from text data.

[0317] A "profile" refers to information that is compiled by analyzing user data and summarizing the user's interests, concerns, emotional state, etc.

[0318] A "virtual environment" is a space generated by computer simulation in which the user's avatar operates.

[0319] An "alter ego" refers to a character in a virtual environment that reflects the user's characteristics and emotional state.

[0320] "Behavioral data" refers to recorded information such as the movements, activities, and interactions of an avatar within a virtual environment.

[0321] "Visualization" refers to displaying analytical results in a visual form, such as a graph or chart, making the data easier to understand.

[0322] The present invention provides a system in which a user interacts with a communication network service, analyzes acquired user data, creates a detailed profile of the user, and generates an avatar of the user in a virtual environment based on the profile. Specific embodiments of this system are described below.

[0323] Data collection

[0324] User: The user launches the dedicated application and clicks a button to connect to a communication network service (e.g., SNS). On the displayed authentication screen, the user enters login information and grants the application permission to access data.

[0325] As a specific example, a user clicks the SNS link button and enters login information on the SNS's OAuth authentication page.

[0326] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data (posts, comments, images, likes, etc.). The server accesses the appropriate API endpoint and collects the user data.

[0327] As a concrete example, the server calls a social networking API to retrieve the user's latest posting data and like history.

[0328] Data analysis

[0329] Server: Preprocesses the acquired user data. Text data is cleaned of unnecessary tags and special characters, and image data is normalized.

[0330] As a specific example, the process involves removing HTML tags contained in text data and deleting emojis and special characters.

[0331] Server: Analyzes text data for keywords, sentiment, and tone using natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER), as well as image analysis (e.g., YOLOv3) to identify user interests.

[0332] As a specific example, a natural language processing model is used to extract frequently used phrases and word tones, and YOLOv3 is used to perform object recognition on image data.

[0333] Server: Using these analysis results, a user profile is generated. The profile includes the user's interests and emotional information. The generated profile is stored in a database.

[0334] As a specific example, the analysis results are compiled in a profile data format and written to a database.

[0335] User avatar generation

[0336] Server: Using the generated profile information as input, a generative AI model (e.g., GPT-3) is used to generate a virtual avatar for the user, which reflects the user's characteristics and emotional state.

[0337] As a specific example, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[0338] Server: The created avatar is placed at the initial position in the virtual environment. The avatar is set to be able to move freely.

[0339] As a specific example, using a virtual environment engine (e.g., Unity), the generated avatar is placed at a specific position in the virtual space and begins to act as an NPC (non-player character).

[0340] Collection of lifestyle data

[0341] Server: The tracking system is activated and the behavior of the avatar in the virtual environment is tracked in real time. The avatar's behavior also reflects emotional information.

[0342] As a specific example, logs of the avatar's movements, communications, and activities are collected in real time and recorded in a database.

[0343] Server: Collects and stores the behavioral data of the avatars in a database. The collected data includes detailed activity logs and emotional states.

[0344] As a specific example, activities performed by an avatar (e.g., visits to a virtual park and interactions with other avatars) are stored as log data.

[0345] Data analysis and visualization

[0346] Server: Analyzes collected behavioral data to identify detailed behavioral patterns, preferences, and emotional states of users.

[0347] As a specific example, the collected data is analyzed using data analysis tools (e.g., Python's Pandas) to identify behavioral patterns and emotional fluctuations.

[0348] Server: The analysis results are visually displayed using a visualization tool (e.g., Tableau), allowing companies to easily understand them and utilize them in their business strategies.

[0349] As a specific example, the analysis results can be displayed on a dashboard as graphs and charts, and can be viewed by companies.

[0350] Here's an example prompt: "Analyze the following social media data (posts, likes, and comments) and generate a user profile. Then, create a virtual avatar of the user based on this profile, track the avatar's behavior, and visualize the data."

[0351] This system allows the creation of detailed user profiles, real-time tracking and analysis of virtual avatar behavior, and the provision of a system that allows companies to effectively utilize the data. Furthermore, the introduction of an emotion engine enables advanced data analysis that reflects the user's emotional state.

[0352] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0353] Step 1: Data collection

[0354] User: The user launches a dedicated application and clicks a button to connect to the communication network service. They enter their login information on the displayed authentication screen and grant the application permission to access data. The input is the user's login information, and the output is an authentication token.

[0355] Specifically, the user clicks the SNS link button and enters their login information on the SNS's OAuth authentication page. This operation obtains an authentication token from the SNS.

[0356] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data. The input is the authentication token and the API request, and the output is user data (posts, comments, images, likes, etc.).

[0357] Specifically, the server calls the SNS API to retrieve the user's latest posts and likes. This data is used in the subsequent analysis steps.

[0358] Step 2: Data Preprocessing

[0359] Server: Preprocesses the acquired user data. The input is raw data and the output is preprocessed data. Data cleaning involves removing unnecessary tags and special characters from text data and normalizing image data.

[0360] Specifically, the system strips out HTML tags from the text data, removes emojis and special characters, and converts the image data into a format suitable for analysis.

[0361] Step 3: Data analysis

[0362] Server: Analyzes the preprocessed data. The input is the preprocessed data, and the output is the analysis results (keywords, sentiment, tone, etc.). It uses natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER) to extract information from text data. It also performs image analysis (e.g., YOLOv3).

[0363] Specifically, the BERT model is applied to text to extract keywords and perform sentiment analysis, and YOLOv3 is used to perform object recognition on image data.

[0364] Step 4: Generate a profile

[0365] Server: Generates a user profile based on the analysis results. The input is the analysis results, and the output is the user profile. The profile includes the user's interests, concerns, and emotional information.

[0366] Specifically, the extracted keywords and emotional information are compiled into profile data format and stored in a database.

[0367] Step 5: Creating a user avatar

[0368] Server: The generated profile information is input into a generative AI model (e.g., GPT-3) to generate the user's avatar in the virtual environment. The input is the user profile, and the output is the user's avatar (virtual character). The avatar reflects the user's characteristics and emotional state.

[0369] Specifically, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[0370] Step 6: Place your clones

[0371] Server: The generated avatar is placed at the initial position in the virtual environment. The input is the generated user avatar, and the output is the avatar in the virtual environment. The avatar is set to be able to act freely.

[0372] Specifically, the generated avatar is placed at a specific position in the virtual space using a virtual environment engine (e.g., Unity). This avatar is designed to interact with other avatars and elements.

[0373] Step 7: Collecting lifestyle data

[0374] Server: Activates the tracking system and tracks the behavior of the avatar in the virtual environment in real time. The input is the activity data of the avatar in the virtual environment, and the output is the behavior log. The behavior of the avatar also includes emotional information.

[0375] Specifically, the system collects logs of the avatar's movements, communications, and activities in real time and records them in a database.

[0376] Step 8: Data analysis and visualization

[0377] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns, preferences, and emotional states. The input is behavioral data, and the output is the analysis results. Data analysis tools (e.g., Python's Pandas) are used to analyze the collected data.

[0378] Specifically, the collected data is analyzed using data analysis tools to identify behavioral patterns and emotional fluctuations.

[0379] Server: Visually displays the analysis results using a visualization tool (e.g., Tableau). The input is the analysis results, and the output is visualized data (graphs, charts, etc.).

[0380] Specifically, the analysis results are displayed on a dashboard as graphs and charts, allowing companies to easily understand the data and use it in their business strategies.

[0381] In this way, the present invention provides a system that generates detailed profiles of users, tracks and analyzes the behavior of their virtual avatars in real time, and allows companies to effectively utilize the data.

[0382] (Application example 2)

[0383] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] Conventional user behavior analysis systems using virtual spaces or social networking services (SNS) have had difficulty providing personalized content that effectively reflects users' interests and emotions. Furthermore, there has been a lack of systems that can collect users' emotional states in real time and use that data to recommend future content. This has limited the improvement of user experience.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0386] In this invention, the server includes: means for a user to link with a social networking service; means for acquiring user data from the social networking service; means for analyzing the user data and creating a user profile; means for generating an avatar of the user in a virtual space based on the profile; means for collecting behavioral data of the user's avatar in the virtual space; means for analyzing the collected behavioral data; means for visualizing the analysis results; means for inserting the generated avatar of the user into video content; means for recommending video content based on the user profile; means for collecting emotional data of the user while watching a video; and means for analyzing the collected emotional data and reflecting it in the next video content recommendation. This makes it possible to provide personalized content that effectively reflects the user's interests and emotions and to improve the user experience.

[0387] A "social networking service" is an online platform that allows users to communicate with each other and share information via the Internet.

[0388] "User Data" refers collectively to user posts, comments, likes, images, and their history information collected on social networking services.

[0389] A "profile" is a set of information generated based on user data, including characteristics such as a user's interests, concerns, behavioral patterns, and emotional state.

[0390] A "virtual space" is a digital world that is generated using computer graphics and other techniques, and in which users can participate interactively.

[0391] A "user's avatar" is a digital avatar that imitates the user and is generated in a virtual space based on the user's profile.

[0392] "Behavioral data" is a record of the actions, movements, and emotional state of the user's avatar in the virtual space.

[0393] "Analysis results" is a general term for insights and information obtained after analyzing collected user data and behavioral data.

[0394] "Visualization" is a technique for visually expressing analytical results in a form that is easy to understand.

[0395] "Video content" is a general term for video and audio distributed over the Internet.

[0396] "Emotional data" is information about a user's emotional state that is analyzed from the user's posts and behavior.

[0397] The following describes an embodiment of the present invention. This system collects and analyzes a user's social networking service (SNS) data to create a user profile and provides personalized video content based on the results. Specifically, it inserts an avatar created using the user data into the video content, and collects emotional data from the user while watching the video to use in recommending the next content.

[0398] The server uses the following hardware and software:

[0399] Hardware:

[0400] Smartphones (including iPhones and Android devices)

[0401] Servers (high-performance computers for analysis and data processing)

[0402] software:

[0403] Python (programming language)

[0404] TensorFlow (a library for implementing deep learning models)

[0405] NLTK (Natural Language Processing Toolkit)

[0406] Django (web framework)

[0407] RESTful API (interface for exchanging data)

[0408] After the user installs the application on their smartphone, the system operates as follows.

[0409] 1. Data Collection:

[0410] A user launches the application and links their social media account. This linking sends data such as the user's posts, comments, and like history to the server using the social media API. For example, a user opens the application, clicks the button to link their Instagram account, and enters their login information on the OAuth authentication screen. This operation retrieves the user's photo posts and comments via the Instagram API.

[0411] 2. Data Analysis:

[0412] The server preprocesses the collected social media data, cleaning the text data and normalizing the image data, then uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from the text data and identify user interests through image analysis.

[0413] 3. User avatar generation:

[0414] Using a generative AI model, the system generates a virtual avatar of the user based on the analyzed profile data, and the avatar can behave in a way that reflects the user's emotions and interests.

[0415] 4. Personalize your video content:

[0416] The server recommends optimal video content based on the user profile. Furthermore, the server inserts the generated avatar into the video and acts as a guide based on the user's interests. For example, if the user likes "travel," the server recommends videos related to travel, in which the user's avatar guides them around tourist spots.

[0417] 5. Collecting Emotional Data:

[0418] While the user is watching the video, the server collects emotional data in real time, which is recorded as post-viewing feedback and used to recommend content for the next time.

[0419] Below are some examples of prompts to use when analyzing collected data:

[0420] Example prompt sentence:

[0421] Extract the following information for a user's Instagram posts:

[0422] keyword

[0423] Emotional state

[0424] Frequent phrases for each keyword

[0425] Based on these prompts, useful information is extracted from the user's posts and used to personalize the video content. The app also takes into account the user's emotional state to recommend the most suitable videos, and the user's avatar acts as a guide within the video, providing a more immersive experience.

[0426] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0427] Program processing flow and detailed explanation of each step

[0428] Step 1:

[0429] Data collection

[0430] Input: The user launches the application and links their social media account.

[0431] Operation: After the user clicks the SNS link button, the device displays the OAuth authentication screen. The user enters their login information and data access permissions.

[0432] Output: SNS authentication token and permissions to access user data are obtained.

[0433] How it works: The server uses the obtained authentication token to send a request to the SNS API to retrieve data such as the user's posts, comments, and likes.

[0434] Step 2:

[0435] Data Preprocessing

[0436] Input: Retrieved social media data.

[0437] How it works: The server cleans the collected data by removing unnecessary tags and special characters, and also normalizes the image data.

[0438] Output: Cleaned text data and normalized image data.

[0439] What it does: Pass this clean data on to the next analysis step.

[0440] Step 3:

[0441] Data analysis

[0442] Input: Cleaned text data and normalized image data.

[0443] How it works: The server uses natural language processing algorithms (e.g., NLTK) and emotion engines to extract keywords, emotional states, and tone from text data. It also performs image analysis to extract features that identify user interests.

[0444] Output: User's keyword list, emotional state and areas of interest.

[0445] How it works: Generates a user profile based on the analysis results.

[0446] Step 4:

[0447] User avatar generation

[0448] Input: The generated user profile.

[0449] How it works: The server uses a generative AI model to generate a virtual avatar for the user based on the user's profile, which reflects the avatar's characteristics and emotional state.

[0450] Output: A virtual avatar of the user.

[0451] Movement: Set the avatar to be able to move freely within the virtual space.

[0452] Step 5:

[0453] Personalized video content

[0454] Input: User profile and virtual self.

[0455] How it works: The server recommends the most suitable video content based on the user profile and inserts the user's avatar into the video. The avatar appears as a navigator in the video, guiding the user through the content.

[0456] Output: Personalized video content.

[0457] What happens: Video is streamed to your device.

[0458] Step 6:

[0459] Collecting Emotional Data

[0460] Input: The video content that the user watches.

[0461] How it works: The device collects the user's emotions in real time while watching, providing post-viewing feedback.

[0462] Output: Real-time collected emotion data.

[0463] Operation: The server analyzes this emotional data and reflects it in the next video content recommendation.

[0464] Step 7:

[0465] Using Feedback

[0466] Input: Collected real-time emotion data.

[0467] How it works: The server analyzes the collected emotional data and reflects it in the next video content recommendation. Specifically, it adjusts the recommendation algorithm based on changes in the user's emotional state and reactions to specific content.

[0468] Output: Upcoming video content recommendation list.

[0469] What it does: Provides users with a list of upcoming video content recommendations.

[0470] Through these steps, the system provides personalized content that effectively reflects the user's interests and emotions.

[0471] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0472] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0473] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0474] [Second embodiment]

[0475] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0476] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0477] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0478] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0479] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0480] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0481] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0482] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0483] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0484] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0485] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0486] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0487] This invention is a system that allows users to connect with social networking services (SNS), analyzes user data obtained from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[0488] Program processing overview

[0489] 1. Data Collection

[0490] Device: The user links their SNS account to the application. At this time, the user enters their SNS account authentication information and grants access rights to the service.

[0491] Server: Calls the SNS API based on the user's authentication information and obtains data such as the user's posts, comments, images, and like history.

[0492] 2. Data Analysis

[0493] Server: Preprocesses the acquired SNS data, for example by cleaning text and normalizing images.

[0494] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data. Also performs image analysis to extract features to identify user interests.

[0495] Server: Generates a user profile based on these analysis results.

[0496] 3. User avatar generation

[0497] Server: Enters profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space.

[0498] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[0499] 4. Collection of lifestyle data

[0500] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[0501] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0502] 5. Data Analysis and Visualization

[0503] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[0504] Server: Visually displays the analysis results using visualization tools, allowing companies to easily understand and utilize the data.

[0505] Specific examples

[0506] Examples of data collection

[0507] Device: The user launches the application and clicks the button to link their Facebook account.

[0508] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[0509] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[0510] Specific examples of data analysis

[0511] Server: In order to analyze the collected data, unnecessary tags and special characters are removed from the text data.

[0512] Server: Uses natural language processing models to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[0513] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[0514] Example of user avatar creation

[0515] Server: Based on the analysis results, a profile of the user is created with characteristics such as "travel lover" or "nature lover."

[0516] Server: This profile is input into the generative AI model, and an avatar of the user is generated in the virtual space.

[0517] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[0518] Examples of collecting lifestyle data

[0519] Server: Records the actions of your avatar in real time, storing logs of the places you visit and the activities you participate in.

[0520] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail and analyzes the user's interests and behavioral patterns.

[0521] Specific examples of data utilization

[0522] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[0523] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[0524] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services.

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] Device: The user launches the application and clicks the button to link their social media account.

[0528] Step 2:

[0529] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[0530] Step 3:

[0531] Device: When the user authorizes the connection, an authentication token is generated.

[0532] Step 4:

[0533] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[0534] Step 5:

[0535] Server: Saves the acquired user data in a database.

[0536] Step 6:

[0537] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[0538] Step 7:

[0539] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[0540] Step 8:

[0541] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[0542] Step 9:

[0543] Server: Creates a user profile based on all analysis results.

[0544] Step 10:

[0545] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[0546] Step 11:

[0547] Server: Places the generated avatar at its initial position in the virtual space.

[0548] Step 12:

[0549] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[0550] Step 13:

[0551] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0552] Step 14:

[0553] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[0554] Step 15:

[0555] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[0556] Step 16:

[0557] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[0558] This allows companies to create detailed profiles of users from their social media data, generate virtual avatars based on those profiles, and track their behavior. The collected data is a very useful source of information for companies.

[0559] Example 1

[0560] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0561] In today's information society, companies and organizations aim to create new value and improve their services by analyzing and utilizing detailed user behavior data. However, with current technology, it has been difficult to efficiently analyze and visualize detailed data, including not only users' online behavior but also their behavioral patterns in virtual environments. Furthermore, technologies for automatically generating and tracking behavior in virtual environments directly from user data have been limited, restricting companies' multifaceted use of data.

[0562] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0563] In this invention, the server includes a means for a user to link with the information sharing system, a means for acquiring user data from the information sharing system, a means for analyzing the user data and creating user characteristics, a means for generating a user representative in a virtual environment based on the characteristics, a means for collecting activity data of the user representative in the virtual environment, a means for analyzing the collected activity data, and a means for visualizing the analysis results. This enables comprehensive analysis of user behavior data online and in the virtual environment, enabling companies to efficiently utilize detailed data.

[0564] "User" refers to an individual or organization that uses the system.

[0565] "Information sharing system" refers to online platforms that provide user data, such as social networking services.

[0566] "User Data" refers to digital information obtained from the information sharing system, such as user posts, comments, images, and like history.

[0567] "Analysis" refers to the process of processing acquired user data and extracting useful information and trends.

[0568] "Traits" refer to information about a user's interests and behavioral patterns generated from analyzed data.

[0569] A "virtual environment" refers to a digital space simulated by a computer.

[0570] "Proxy" refers to a digital avatar that is generated to simulate the user's actions within a virtual environment.

[0571] "Activity Data" refers to a record of actions taken by an agent within a virtual environment.

[0572] "Visualization" refers to the process of displaying analytical results in the form of diagrams, charts, etc., to make the data easier to understand.

[0573] MODE FOR CARRYING OUT THE INVENTION

[0574] The present invention is a system that allows users to connect to an information sharing system, analyzes the user data acquired from the system, creates user characteristics, and generates a user's agent in a virtual environment. It also provides a platform that collects and analyzes the agent's activity data in the virtual environment and enables companies to utilize detailed data.

[0575] 1. Data collection and collaboration

[0576] Terminal: To access the information sharing system, a user launches an application on the terminal and clicks, for example, a "Log in with Facebook" button. The user enters their login information and grants the necessary data access permissions to link with the information sharing system.

[0577] Server: Using the obtained authentication token, the server calls the API of the information sharing system and obtains data such as the user's posts, comments, images, and like history. For example, the server obtains user data using the Facebook API.

[0578] 2. Data Analysis

[0579] Server: Preprocesses the collected data. For example, it uses Python regular expressions to remove unnecessary HTML tags and special characters from text data. Similarly, it standardizes the format and size of image data.

[0580] Server: Uses natural language processing algorithms to extract keywords, emotions, and tones from text data. For example, it uses Python's NLTK library to classify positive and negative emotions. It also performs image analysis using OpenCV to recognize objects and scenes in images.

[0581] Server: Based on the results of these analyses, the server generates user characteristics, including user interests and behavioral patterns.

[0582] 3. Creating a User Agent

[0583] Server: Input user characteristics into the generative AI model and generate a representative with the user's characteristics in the virtual environment. For example, input the prompt "Generate a user who likes to travel in the virtual space" into the generative AI model (e.g., GPT-3).

[0584] Example prompt:

[0585] Create a virtual travel-loving user who is a nature lover and travels frequently.

[0586] Server: Places the generated agent at an initial position in the virtual environment and allows the agent to move freely. Using the virtual environment engine, imports a 3D model and performs initial settings.

[0587] 4. Agent Activity Data Collection

[0588] Server: Activates the tracking system and tracks the agent's actions in real time, for example, recording the time, location, and interactions of the agent with other users in the virtual park.

[0589] Server: The agent's movements and activities are stored in a database as log data.

[0590] 5. Data Analysis and Visualization

[0591] Server: Analyzes the collected behavioral data to identify user behavioral patterns and preferences. Classifies the agent's behavior using clustering techniques and visually analyzes the data.

[0592] Server: Displays the analysis results using visualization tools (e.g., Tableau or Power BI), allowing companies to easily understand and utilize the data.

[0593] This invention enables comprehensive analysis of users' online and virtual behavioral data, enabling companies to efficiently utilize detailed data. This system contributes to the creation of new value and the improvement of services.

[0594] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0595] Step 1: Data collection

[0596] Input: Credentials for users to log in to the information sharing system

[0597] What happens: A user launches the application and clicks the "Log in with Facebook" button. The user enters their login information in the OAuth authentication screen that appears and grants the application the necessary data access permissions.

[0598] Processing: An OAuth token is obtained based on the authentication information, and the server uses the information sharing system's API to obtain data such as the user's posts, comments, images, and like history.

[0599] Output: Retrieved user data

[0600] Step 2: Data Preprocessing

[0601] Input: Acquired user data (text, images, etc.)

[0602] Specific operations: The server removes unnecessary HTML tags and special characters from text data, performs preprocessing such as using Python regular expressions, and standardizes the format and size of image data.

[0603] Processing: Clean and normalize the data.

[0604] Output: Preprocessed user data

[0605] Step 3: Data analysis

[0606] Input: Preprocessed user data (text, images, etc.)

[0607] How it works: The server uses natural language processing algorithms (e.g., Python's NLTK library) to extract keywords, emotions, and tones from text data. The server uses image analysis techniques (e.g., OpenCV) to recognize objects and scenes in images.

[0608] Processing: Extract keywords and emotions from text data, and recognize objects and scenes from image data.

[0609] Output: User characteristics as analysis results

[0610] Step 4: Creating a User Agent

[0611] Input: User characteristics

[0612] Specific operation: The server creates a prompt for the generated AI model (e.g., GPT-3) to input the user's characteristic information and inputs it into the model. The specific prompt is, "Please generate a user who loves to travel in the virtual space. This user is a nature lover and travels frequently."

[0613] Processing: A generative AI model generates a user representative in a virtual space based on input prompts.

[0614] Output: A user representative generated in the virtual space

[0615] Step 5: Collecting data on agent activities

[0616] Input: A user's representative generated in the virtual space

[0617] Specific operation: The server starts a tracking system to track the agent's actions in real time, recording the agent's location, movement route, and interactions with other agents.

[0618] Processing: Collecting agent behavior data and storing it in a database.

[0619] Output: Detailed agent behavior data

[0620] Step 6: Data analysis and visualization

[0621] Input: Detailed agent behavior data

[0622] Specific operations: Analyze the behavioral data collected by the server to identify behavioral patterns and preferences. Classify the behavioral data using clustering techniques. Display the analysis results using a visualization tool (e.g., Tableau or Power BI).

[0623] Processing: Analysis and visualization of behavioral data

[0624] Output: Analyzed behavioral patterns and preferences, visualized data

[0625] In this way, by having the user and server perform specific actions at each step, the system can perform detailed analysis of user behavior data and provide data that companies can use to create new value.

[0626] (Application example 1)

[0627] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0628] Conventional virtual user avatar systems have the problem that they do not adequately provide personalized product recommendations based on the user's interests. In addition, the collection and analysis of user behavior data in virtual space is insufficient, making it difficult for companies to obtain information that is useful for marketing strategies and new product development.

[0629] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0630] In this invention, the server includes means for allowing a user to link with a social networking service, means for acquiring user data from the social networking service, means for analyzing the user data and creating a user profile, means for generating an avatar of the user in a virtual space based on the profile, means for collecting behavioral data of the user's avatar in the virtual space, means for analyzing the collected behavioral data, means for visualizing the analysis results, and means for making product suggestions to the user in a virtual store based on the user's profile. This enables personalized product suggestions based on the user's interests and behavioral patterns, and allows companies to obtain detailed data for efficient and effective marketing strategies and new product development.

[0631] A "social networking service" is an online platform that enables users to connect, interact, and share information with other users over the Internet.

[0632] "User Data" refers collectively to information generated, shared, or engaged with by users on social networking services, including posts, comments, images, and like history.

[0633] A "virtual space" is an artificial three-dimensional space generated by a computer, in which users can interact through avatars.

[0634] An "alter ego" is an avatar generated in a virtual space based on a user's profile, and is a character that acts in the virtual space reflecting the user's actions, interests, and concerns.

[0635] A "virtual store" is a virtual store operated within a virtual space, where users can view and purchase virtual products.

[0636] "Personalized product suggestions" refers to the suggestion of individually customized products and services based on each user's interests and behavioral patterns.

[0637] "Natural language processing algorithms" are computational techniques for understanding, analyzing, and generating human language, and are used to extract meaning and sentiment from text data.

[0638] "Visualization" refers to the visual representation of data, making the results of data analysis intuitively understandable through dashboards, graphs, charts, etc.

[0639] This invention is a system that allows users to link with social networking services (SNS), analyzes user data acquired from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[0640] This system is realized with the following configuration.

[0641] Data collection

[0642] The user connects to their SNS account using their device. They enter their SNS account authentication information and grant access to the service. The server then calls the SNS API based on the user's authentication information to retrieve data such as the user's posts, comments, images, and like history.

[0643] Data analysis

[0644] The server preprocesses the acquired social media data, for example by cleaning the text and normalizing the images. It then uses natural language processing algorithms to extract keywords, emotions, and tones from the text data, and image analysis techniques to recognize objects and scenes in the images. Based on these analysis results, the server generates a user profile.

[0645] User avatar generation

[0646] The server inputs the user's profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space. The generated avatar is placed in an initial position in the virtual space and can move freely.

[0647] Collection of lifestyle data

[0648] The server starts a tracking system and tracks the actions of the avatar in real time, recording the avatar's movements and activities as log data and saving it in a database.

[0649] Data analysis and visualization

[0650] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, making it easy for companies to understand and utilize the data.

[0651] Product proposals in virtual stores

[0652] In a virtual store in a virtual space, a user's avatar interacts with other avatars, and personalized product suggestions are made based on their behavioral data, allowing for product suggestions based on the user's interests and behavioral patterns.

[0653] The main hardware required to realize this system is a smartphone, and the following software is used:

[0654] Platform: Android / iOS

[0655] Language: Java (Android), Swift (iOS)

[0656] API: Social networking service API (e.g. Facebook Graph API)

[0657] Libraries: Gson (JSON parsing), OkHttp3 (HTTP communication)

[0658] Generative AI model: Used to generate avatars based on user profiles

[0659] As a concrete example, the following prompt sentence can be used:

[0660] Example prompt sentence:

[0661] Generate an active user avatar in a virtual space based on the user's social media data (interests: travel, activity: hiking). Collect simulation data of the avatar searching for travel-related goods and activities in a virtual store. Display this data on a visualization dashboard, allowing travel-related companies to use it to propose new products.

[0662] This will enable companies to gain a detailed understanding of users' preferences and behavioral patterns, enabling them to propose more personalized products.

[0663] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0664] Step 1:

[0665] A user connects to a social networking account using a device. The user launches the application, enters their social networking account authentication information, and grants access to the service. At this time, the device sends the authentication information to the server and receives an authentication token to call the SNS API. The output is an authentication token.

[0666] Step 2:

[0667] The server uses the authentication token to call the SNS API and obtain data such as the user's posts, comments, images, and like history. This data is obtained in JSON format and temporarily stored on the server. The input is the authentication token, and the output is the user's SNS data.

[0668] Step 3:

[0669] The server preprocesses the acquired social media data. Specifically, it cleans the text (removes unnecessary tags and special characters) and normalizes the images. It uses natural language processing algorithms to extract keywords, emotions, and tones from the text data. It also uses image analysis techniques to recognize objects and scenes in the images. The input is the social media data, and the output is the analysis results.

[0670] Step 4:

[0671] The server generates a user profile based on the analysis results. This profile contains detailed information about the user, such as their interests and behavioral patterns. The profile is stored in a database. The input is the analysis results, and the output is the user profile.

[0672] Step 5:

[0673] The server inputs the user's profile information into the generative AI model and generates the user's avatar in the virtual space. The generated avatar is placed at an initial position in the virtual space and set up so that it can move freely. The input is the user profile, and the output is the user's avatar.

[0674] Step 6:

[0675] The server starts the tracking system and tracks the avatar's actions in real time. The avatar's movements and activities are recorded as log data and stored in a database. The input is the user's avatar, and the output is the behavior log.

[0676] Step 7:

[0677] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, allowing companies to easily understand and utilize the data. The input is the behavior log, and the output is the visualized analysis results.

[0678] Step 8:

[0679] In a virtual store in a virtual space, the user's avatar interacts with other avatars. The server uses this behavioral data to make personalized product suggestions. It identifies, highlights, and suggests products and services that the user may be interested in. The input is the behavioral log and user profile, and the output is product suggestions.

[0680] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0681] This invention is a system that allows users to link with social networking services (SNS), analyzes SNS data to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, by combining it with an emotion engine, we provide a platform that recognizes and analyzes the user's emotions and reflects that emotional information in the profile and behavioral data.

[0682] Program processing overview

[0683] 1. Data Collection

[0684] On the device: The user launches the application and clicks the button to link their SNS account. The user enters their login information for the SNS account and grants the app permission to access their data.

[0685] Server: Sends a request to the SNS API using the authentication token to retrieve user data (posts, comments, images, likes, etc.).

[0686] 2. Data Analysis

[0687] Server: Preprocesses the collected SNS data, cleaning text data and normalizing image data.

[0688] Server: Uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from text data. It also performs image analysis to extract features to identify user interests.

[0689] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[0690] 3. User avatar generation

[0691] Server: Enters profile information into the generative AI model and generates an avatar in the virtual space that reflects the user's characteristics and emotional state.

[0692] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[0693] 4. Collection of lifestyle data

[0694] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[0695] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0696] 5. Data Analysis and Visualization

[0697] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns and preferences, as well as emotional information.

[0698] Server: Visually displays the analysis results using visualization tools, making it easy for companies to understand and utilize the data.

[0699] Specific examples

[0700] Examples of data collection

[0701] Device: The user launches the application and clicks the button to link their Facebook account.

[0702] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[0703] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[0704] Specific examples of data analysis

[0705] Server: Preprocesses the acquired data and removes unnecessary tags and special characters from the text data.

[0706] Server: Uses natural language processing models and sentiment engines to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[0707] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[0708] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[0709] Example of user avatar creation

[0710] Server: Based on the analysis results, a user profile is created with characteristics and emotional information such as "travel lover" or "nature lover."

[0711] Server: This profile is input into a generative AI model to generate an avatar of the user in a virtual space. The avatar can then behave in a way that reflects the user's emotional state.

[0712] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[0713] Examples of collecting lifestyle data

[0714] Server: Records the actions of the avatar in real time, storing logs of the places visited and the activities participated in. The actions of the avatar also include emotional information.

[0715] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail, analyzes the user's interests and behavioral patterns, and also records emotional information.

[0716] Specific examples of data utilization

[0717] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[0718] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[0719] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services. The addition of an emotion engine makes it possible to collect detailed data that reflects user emotions, resulting in more accurate analysis results.

[0720] The processing flow will be explained below.

[0721] Step 1:

[0722] Device: The user launches the application and clicks the button to link their social media account.

[0723] Step 2:

[0724] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[0725] Step 3:

[0726] Device: When the user authorizes the connection, an authentication token is generated.

[0727] Step 4:

[0728] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[0729] Step 5:

[0730] Server: Saves the acquired user data in a database.

[0731] Step 6:

[0732] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[0733] Step 7:

[0734] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[0735] Step 8:

[0736] Server: Analyzes and recognizes the user's emotional information using the emotion engine.

[0737] Step 9:

[0738] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[0739] Step 10:

[0740] Server: Creates a user profile based on all analysis results, including emotional information.

[0741] Step 11:

[0742] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[0743] Step 12:

[0744] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[0745] Step 13:

[0746] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[0747] Step 14:

[0748] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0749] Step 15:

[0750] Server: Analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences, including emotional information.

[0751] Step 16:

[0752] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[0753] Step 17:

[0754] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[0755] This allows for the creation of a detailed profile from a user's social media data, the generation of an avatar based on that profile in a virtual space, and tracking of their behavior. The addition of an emotion engine makes it possible to collect and analyze detailed data that reflects user emotions, making it an extremely useful source of information for companies.

[0756] Example 2

[0757] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0758] In recent years, the number of users of social networking services has increased, leading to a diversification of the analysis and utilization of user data. However, conventional systems have had difficulty generating detailed profiles of users' emotions and behaviors, and tracking and analyzing their behavior in virtual spaces in real time. Furthermore, there are limited means for visually displaying the analysis results and for companies to effectively utilize them. Therefore, there is a need for a new system that can provide advanced analysis results, generate detailed profiles based on users' emotions and behaviors, and effectively utilize them in business strategies.

[0759] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0760] In this invention, the server includes a means for a user to link with a communication network service, a means for acquiring user data from the communication network service, a means for preprocessing the acquired user data, a means for analyzing the user data and creating a user profile, a means for generating an avatar of the user in a virtual environment based on the profile, a means for collecting behavioral data of the user's avatar in the virtual environment, a means for analyzing the collected behavioral data, and a means for visualizing the analysis results. This makes it possible to generate a detailed user profile and track and analyze behavior in a virtual space, thereby providing a system that can effectively utilize the collected data.

[0761] "User" refers to any individual or corporation that uses this system.

[0762] "Communication network services" is a general term for services that allow users to exchange information via the Internet.

[0763] "User Data" refers to all information generated by users of communication network services, including posts, comments, images, like history, etc.

[0764] "Preprocessing" refers to the process of formatting and cleaning up data before analysis, and includes removing unnecessary tags and special characters, normalizing image data, etc.

[0765] "Language analysis algorithm" refers to an algorithm that uses natural language processing technology to extract useful information (keywords, emotions, tone, etc.) from text data.

[0766] A "profile" refers to information that is compiled by analyzing user data and summarizing the user's interests, concerns, emotional state, etc.

[0767] A "virtual environment" is a space generated by computer simulation in which the user's avatar operates.

[0768] An "alter ego" refers to a character in a virtual environment that reflects the user's characteristics and emotional state.

[0769] "Behavioral data" refers to recorded information such as the movements, activities, and interactions of an avatar within a virtual environment.

[0770] "Visualization" refers to displaying analytical results in a visual form, such as a graph or chart, making the data easier to understand.

[0771] The present invention provides a system in which a user interacts with a communication network service, analyzes acquired user data, creates a detailed profile of the user, and generates an avatar of the user in a virtual environment based on the profile. Specific embodiments of this system are described below.

[0772] Data collection

[0773] User: The user launches the dedicated application and clicks a button to connect to a communication network service (e.g., SNS). On the displayed authentication screen, the user enters login information and grants the application permission to access data.

[0774] As a specific example, a user clicks the SNS link button and enters login information on the SNS's OAuth authentication page.

[0775] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data (posts, comments, images, likes, etc.). The server accesses the appropriate API endpoint and collects the user data.

[0776] As a concrete example, the server calls a social networking API to retrieve the user's latest posting data and like history.

[0777] Data analysis

[0778] Server: Preprocesses the acquired user data. Text data is cleaned of unnecessary tags and special characters, and image data is normalized.

[0779] As a specific example, the process involves removing HTML tags contained in text data and deleting emojis and special characters.

[0780] Server: Analyzes text data for keywords, sentiment, and tone using natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER), as well as image analysis (e.g., YOLOv3) to identify user interests.

[0781] As a specific example, a natural language processing model is used to extract frequently used phrases and word tones, and YOLOv3 is used to perform object recognition on image data.

[0782] Server: Using these analysis results, a user profile is generated. The profile includes the user's interests and emotional information. The generated profile is stored in a database.

[0783] As a specific example, the analysis results are compiled in a profile data format and written to a database.

[0784] User avatar generation

[0785] Server: Using the generated profile information as input, a generative AI model (e.g., GPT-3) is used to generate a virtual avatar for the user, which reflects the user's characteristics and emotional state.

[0786] As a specific example, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[0787] Server: The created avatar is placed at the initial position in the virtual environment. The avatar is set to be able to move freely.

[0788] As a specific example, using a virtual environment engine (e.g., Unity), the generated avatar is placed at a specific position in the virtual space and begins to act as an NPC (non-player character).

[0789] Collection of lifestyle data

[0790] Server: The tracking system is activated and the behavior of the avatar in the virtual environment is tracked in real time. The avatar's behavior also reflects emotional information.

[0791] As a specific example, logs of the avatar's movements, communications, and activities are collected in real time and recorded in a database.

[0792] Server: Collects and stores the behavioral data of the avatars in a database. The collected data includes detailed activity logs and emotional states.

[0793] As a specific example, activities performed by an avatar (e.g., visits to a virtual park and interactions with other avatars) are stored as log data.

[0794] Data analysis and visualization

[0795] Server: Analyzes collected behavioral data to identify detailed behavioral patterns, preferences, and emotional states of users.

[0796] As a specific example, the collected data is analyzed using data analysis tools (e.g., Python's Pandas) to identify behavioral patterns and emotional fluctuations.

[0797] Server: The analysis results are visually displayed using a visualization tool (e.g., Tableau), allowing companies to easily understand them and utilize them in their business strategies.

[0798] As a specific example, the analysis results can be displayed on a dashboard as graphs and charts, and can be viewed by companies.

[0799] Here's an example prompt: "Analyze the following social media data (posts, likes, and comments) and generate a user profile. Then, create a virtual avatar of the user based on this profile, track the avatar's behavior, and visualize the data."

[0800] This system allows the creation of detailed user profiles, real-time tracking and analysis of virtual avatar behavior, and the provision of a system that allows companies to effectively utilize the data. Furthermore, the introduction of an emotion engine enables advanced data analysis that reflects the user's emotional state.

[0801] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0802] Step 1: Data collection

[0803] User: The user launches a dedicated application and clicks a button to connect to the communication network service. They enter their login information on the displayed authentication screen and grant the application permission to access data. The input is the user's login information, and the output is an authentication token.

[0804] Specifically, the user clicks the SNS link button and enters their login information on the SNS's OAuth authentication page. This operation obtains an authentication token from the SNS.

[0805] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data. The input is the authentication token and the API request, and the output is user data (posts, comments, images, likes, etc.).

[0806] Specifically, the server calls the SNS API to retrieve the user's latest posts and likes. This data is used in the subsequent analysis steps.

[0807] Step 2: Data Preprocessing

[0808] Server: Preprocesses the acquired user data. The input is raw data and the output is preprocessed data. Data cleaning involves removing unnecessary tags and special characters from text data and normalizing image data.

[0809] Specifically, the system strips out HTML tags from the text data, removes emojis and special characters, and converts the image data into a format suitable for analysis.

[0810] Step 3: Data analysis

[0811] Server: Analyzes the preprocessed data. The input is the preprocessed data, and the output is the analysis results (keywords, sentiment, tone, etc.). It uses natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER) to extract information from text data. It also performs image analysis (e.g., YOLOv3).

[0812] Specifically, the BERT model is applied to text to extract keywords and perform sentiment analysis, and YOLOv3 is used to perform object recognition on image data.

[0813] Step 4: Generate a profile

[0814] Server: Generates a user profile based on the analysis results. The input is the analysis results, and the output is the user profile. The profile includes the user's interests, concerns, and emotional information.

[0815] Specifically, the extracted keywords and emotional information are compiled into profile data format and stored in a database.

[0816] Step 5: Creating a user avatar

[0817] Server: The generated profile information is input into a generative AI model (e.g., GPT-3) to generate the user's avatar in the virtual environment. The input is the user profile, and the output is the user's avatar (virtual character). The avatar reflects the user's characteristics and emotional state.

[0818] Specifically, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[0819] Step 6: Place your clones

[0820] Server: The generated avatar is placed at the initial position in the virtual environment. The input is the generated user avatar, and the output is the avatar in the virtual environment. The avatar is set to be able to act freely.

[0821] Specifically, the generated avatar is placed at a specific position in the virtual space using a virtual environment engine (e.g., Unity). This avatar is designed to interact with other avatars and elements.

[0822] Step 7: Collecting lifestyle data

[0823] Server: Activates the tracking system and tracks the behavior of the avatar in the virtual environment in real time. The input is the activity data of the avatar in the virtual environment, and the output is the behavior log. The behavior of the avatar also includes emotional information.

[0824] Specifically, the system collects logs of the avatar's movements, communications, and activities in real time and records them in a database.

[0825] Step 8: Data analysis and visualization

[0826] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns, preferences, and emotional states. The input is behavioral data, and the output is the analysis results. Data analysis tools (e.g., Python's Pandas) are used to analyze the collected data.

[0827] Specifically, the collected data is analyzed using data analysis tools to identify behavioral patterns and emotional fluctuations.

[0828] Server: Visually displays the analysis results using a visualization tool (e.g., Tableau). The input is the analysis results, and the output is visualized data (graphs, charts, etc.).

[0829] Specifically, the analysis results are displayed on a dashboard as graphs and charts, allowing companies to easily understand the data and use it in their business strategies.

[0830] In this way, the present invention provides a system that generates detailed profiles of users, tracks and analyzes the behavior of their virtual avatars in real time, and allows companies to effectively utilize the data.

[0831] (Application example 2)

[0832] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0833] Conventional user behavior analysis systems using virtual spaces or social networking services (SNS) have had difficulty providing personalized content that effectively reflects users' interests and emotions. Furthermore, there has been a lack of systems that can collect users' emotional states in real time and use that data to recommend future content. This has limited the improvement of user experience.

[0834] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0835] In this invention, the server includes: means for a user to link with a social networking service; means for acquiring user data from the social networking service; means for analyzing the user data and creating a user profile; means for generating an avatar of the user in a virtual space based on the profile; means for collecting behavioral data of the user's avatar in the virtual space; means for analyzing the collected behavioral data; means for visualizing the analysis results; means for inserting the generated avatar of the user into video content; means for recommending video content based on the user profile; means for collecting emotional data of the user while watching a video; and means for analyzing the collected emotional data and reflecting it in the next video content recommendation. This makes it possible to provide personalized content that effectively reflects the user's interests and emotions and to improve the user experience.

[0836] A "social networking service" is an online platform that allows users to communicate with each other and share information via the Internet.

[0837] "User Data" refers collectively to user posts, comments, likes, images, and their history information collected on social networking services.

[0838] A "profile" is a set of information generated based on user data, including characteristics such as a user's interests, concerns, behavioral patterns, and emotional state.

[0839] A "virtual space" is a digital world that is generated using computer graphics and other techniques, and in which users can participate interactively.

[0840] A "user's avatar" is a digital avatar that imitates the user and is generated in a virtual space based on the user's profile.

[0841] "Behavioral data" is a record of the actions, movements, and emotional state of the user's avatar in the virtual space.

[0842] "Analysis results" is a general term for insights and information obtained after analyzing collected user data and behavioral data.

[0843] "Visualization" is a technique for visually expressing analytical results in a form that is easy to understand.

[0844] "Video content" is a general term for video and audio distributed over the Internet.

[0845] "Emotional data" is information about a user's emotional state that is analyzed from the user's posts and behavior.

[0846] The following describes an embodiment of the present invention. This system collects and analyzes a user's social networking service (SNS) data to create a user profile and provides personalized video content based on the results. Specifically, it inserts an avatar created using the user data into the video content, and collects emotional data from the user while watching the video to use in recommending the next content.

[0847] The server uses the following hardware and software:

[0848] Hardware:

[0849] Smartphones (including iPhones and Android devices)

[0850] Servers (high-performance computers for analysis and data processing)

[0851] software:

[0852] Python (programming language)

[0853] TensorFlow (a library for implementing deep learning models)

[0854] NLTK (Natural Language Processing Toolkit)

[0855] Django (web framework)

[0856] RESTful API (interface for exchanging data)

[0857] After the user installs the application on their smartphone, the system operates as follows.

[0858] 1. Data Collection:

[0859] A user launches the application and links their social media account. This linking sends data such as the user's posts, comments, and like history to the server using the social media API. For example, a user opens the application, clicks the button to link their Instagram account, and enters their login information on the OAuth authentication screen. This operation retrieves the user's photo posts and comments via the Instagram API.

[0860] 2. Data Analysis:

[0861] The server preprocesses the collected social media data, cleaning the text data and normalizing the image data, then uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from the text data and identify user interests through image analysis.

[0862] 3. User avatar generation:

[0863] Using a generative AI model, the system generates a virtual avatar of the user based on the analyzed profile data, and the avatar can behave in a way that reflects the user's emotions and interests.

[0864] 4. Personalize your video content:

[0865] The server recommends optimal video content based on the user profile. Furthermore, the server inserts the generated avatar into the video and acts as a guide based on the user's interests. For example, if the user likes "travel," the server recommends videos related to travel, in which the user's avatar guides them around tourist spots.

[0866] 5. Collecting Emotional Data:

[0867] While the user is watching the video, the server collects emotional data in real time, which is recorded as post-viewing feedback and used to recommend content for the next time.

[0868] Below are some examples of prompts to use when analyzing collected data:

[0869] Example prompt sentence:

[0870] Extract the following information for a user's Instagram posts:

[0871] keyword

[0872] Emotional state

[0873] Frequent phrases for each keyword

[0874] Based on these prompts, useful information is extracted from the user's posts and used to personalize the video content. The app also takes into account the user's emotional state to recommend the most suitable videos, and the user's avatar acts as a guide within the video, providing a more immersive experience.

[0875] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0876] Program processing flow and detailed explanation of each step

[0877] Step 1:

[0878] Data collection

[0879] Input: The user launches the application and links their social media account.

[0880] Operation: After the user clicks the SNS link button, the device displays the OAuth authentication screen. The user enters their login information and data access permissions.

[0881] Output: SNS authentication token and permissions to access user data are obtained.

[0882] How it works: The server uses the obtained authentication token to send a request to the SNS API to retrieve data such as the user's posts, comments, and likes.

[0883] Step 2:

[0884] Data Preprocessing

[0885] Input: Retrieved social media data.

[0886] How it works: The server cleans the collected data by removing unnecessary tags and special characters, and also normalizes the image data.

[0887] Output: Cleaned text data and normalized image data.

[0888] What it does: Pass this clean data on to the next analysis step.

[0889] Step 3:

[0890] Data analysis

[0891] Input: Cleaned text data and normalized image data.

[0892] How it works: The server uses natural language processing algorithms (e.g., NLTK) and emotion engines to extract keywords, emotional states, and tone from text data. It also performs image analysis to extract features that identify user interests.

[0893] Output: User's keyword list, emotional state and areas of interest.

[0894] How it works: Generates a user profile based on the analysis results.

[0895] Step 4:

[0896] User avatar generation

[0897] Input: The generated user profile.

[0898] How it works: The server uses a generative AI model to generate a virtual avatar for the user based on the user's profile, which reflects the avatar's characteristics and emotional state.

[0899] Output: A virtual avatar of the user.

[0900] Movement: Set the avatar to be able to move freely within the virtual space.

[0901] Step 5:

[0902] Personalized video content

[0903] Input: User profile and virtual self.

[0904] How it works: The server recommends the most suitable video content based on the user profile and inserts the user's avatar into the video. The avatar appears as a navigator in the video, guiding the user through the content.

[0905] Output: Personalized video content.

[0906] What happens: Video is streamed to your device.

[0907] Step 6:

[0908] Collecting Emotional Data

[0909] Input: The video content that the user watches.

[0910] How it works: The device collects the user's emotions in real time while watching, providing post-viewing feedback.

[0911] Output: Real-time collected emotion data.

[0912] Operation: The server analyzes this emotional data and reflects it in the next video content recommendation.

[0913] Step 7:

[0914] Using Feedback

[0915] Input: Collected real-time emotion data.

[0916] How it works: The server analyzes the collected emotional data and reflects it in the next video content recommendation. Specifically, it adjusts the recommendation algorithm based on changes in the user's emotional state and reactions to specific content.

[0917] Output: Upcoming video content recommendation list.

[0918] What it does: Provides users with a list of upcoming video content recommendations.

[0919] Through these steps, the system provides personalized content that effectively reflects the user's interests and emotions.

[0920] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0921] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0922] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0923] [Third embodiment]

[0924] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0925] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0926] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0927] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0928] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0929] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0930] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0931] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0932] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0933] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0934] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0935] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0936] This invention is a system that allows users to connect with social networking services (SNS), analyzes user data obtained from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[0937] Program processing overview

[0938] 1. Data Collection

[0939] Device: The user links their SNS account to the application. At this time, the user enters their SNS account authentication information and grants access rights to the service.

[0940] Server: Calls the SNS API based on the user's authentication information and obtains data such as the user's posts, comments, images, and like history.

[0941] 2. Data Analysis

[0942] Server: Preprocesses the acquired SNS data, for example by cleaning text and normalizing images.

[0943] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data. Also performs image analysis to extract features to identify user interests.

[0944] Server: Generates a user profile based on these analysis results.

[0945] 3. User avatar generation

[0946] Server: Enters profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space.

[0947] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[0948] 4. Collection of lifestyle data

[0949] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[0950] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[0951] 5. Data Analysis and Visualization

[0952] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[0953] Server: Visually displays the analysis results using visualization tools, allowing companies to easily understand and utilize the data.

[0954] Specific examples

[0955] Examples of data collection

[0956] Device: The user launches the application and clicks the button to link their Facebook account.

[0957] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[0958] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[0959] Specific examples of data analysis

[0960] Server: In order to analyze the collected data, unnecessary tags and special characters are removed from the text data.

[0961] Server: Uses natural language processing models to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[0962] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[0963] Example of user avatar creation

[0964] Server: Based on the analysis results, a profile of the user is created with characteristics such as "travel lover" or "nature lover."

[0965] Server: This profile is input into the generative AI model, and an avatar of the user is generated in the virtual space.

[0966] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[0967] Examples of collecting lifestyle data

[0968] Server: Records the actions of your avatar in real time, storing logs of the places you visit and the activities you participate in.

[0969] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail and analyzes the user's interests and behavioral patterns.

[0970] Specific examples of data utilization

[0971] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[0972] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[0973] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services.

[0974] The processing flow will be explained below.

[0975] Step 1:

[0976] Device: The user launches the application and clicks the button to link their social media account.

[0977] Step 2:

[0978] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[0979] Step 3:

[0980] Device: When the user authorizes the connection, an authentication token is generated.

[0981] Step 4:

[0982] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[0983] Step 5:

[0984] Server: Saves the acquired user data in a database.

[0985] Step 6:

[0986] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[0987] Step 7:

[0988] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[0989] Step 8:

[0990] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[0991] Step 9:

[0992] Server: Creates a user profile based on all analysis results.

[0993] Step 10:

[0994] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[0995] Step 11:

[0996] Server: Places the generated avatar at its initial position in the virtual space.

[0997] Step 12:

[0998] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[0999] Step 13:

[1000] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[1001] Step 14:

[1002] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[1003] Step 15:

[1004] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[1005] Step 16:

[1006] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[1007] This allows companies to create detailed profiles of users from their social media data, generate virtual avatars based on those profiles, and track their behavior. The collected data is a very useful source of information for companies.

[1008] Example 1

[1009] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] In today's information society, companies and organizations aim to create new value and improve their services by analyzing and utilizing detailed user behavior data. However, with current technology, it has been difficult to efficiently analyze and visualize detailed data, including not only users' online behavior but also their behavioral patterns in virtual environments. Furthermore, technologies for automatically generating and tracking behavior in virtual environments directly from user data have been limited, restricting companies' multifaceted use of data.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1012] In this invention, the server includes a means for a user to link with the information sharing system, a means for acquiring user data from the information sharing system, a means for analyzing the user data and creating user characteristics, a means for generating a user representative in a virtual environment based on the characteristics, a means for collecting activity data of the user representative in the virtual environment, a means for analyzing the collected activity data, and a means for visualizing the analysis results. This enables comprehensive analysis of user behavior data online and in the virtual environment, enabling companies to efficiently utilize detailed data.

[1013] "User" refers to an individual or organization that uses the system.

[1014] "Information sharing system" refers to online platforms that provide user data, such as social networking services.

[1015] "User Data" refers to digital information obtained from the information sharing system, such as user posts, comments, images, and like history.

[1016] "Analysis" refers to the process of processing acquired user data and extracting useful information and trends.

[1017] "Traits" refer to information about a user's interests and behavioral patterns generated from analyzed data.

[1018] A "virtual environment" refers to a digital space simulated by a computer.

[1019] "Proxy" refers to a digital avatar that is generated to simulate the user's actions within a virtual environment.

[1020] "Activity Data" refers to a record of actions taken by an agent within a virtual environment.

[1021] "Visualization" refers to the process of displaying analytical results in the form of diagrams, charts, etc., to make the data easier to understand.

[1022] MODE FOR CARRYING OUT THE INVENTION

[1023] The present invention is a system that allows users to connect to an information sharing system, analyzes the user data acquired from the system, creates user characteristics, and generates a user's agent in a virtual environment. It also provides a platform that collects and analyzes the agent's activity data in the virtual environment and enables companies to utilize detailed data.

[1024] 1. Data collection and collaboration

[1025] Terminal: To access the information sharing system, a user launches an application on the terminal and clicks, for example, a "Log in with Facebook" button. The user enters their login information and grants the necessary data access permissions to link with the information sharing system.

[1026] Server: Using the obtained authentication token, the server calls the API of the information sharing system and obtains data such as the user's posts, comments, images, and like history. For example, the server obtains user data using the Facebook API.

[1027] 2. Data Analysis

[1028] Server: Preprocesses the collected data. For example, it uses Python regular expressions to remove unnecessary HTML tags and special characters from text data. Similarly, it standardizes the format and size of image data.

[1029] Server: Uses natural language processing algorithms to extract keywords, emotions, and tones from text data. For example, it uses Python's NLTK library to classify positive and negative emotions. It also performs image analysis using OpenCV to recognize objects and scenes in images.

[1030] Server: Based on the results of these analyses, the server generates user characteristics, including user interests and behavioral patterns.

[1031] 3. Creating a User Agent

[1032] Server: Input user characteristics into the generative AI model and generate a representative with the user's characteristics in the virtual environment. For example, input the prompt "Generate a user who likes to travel in the virtual space" into the generative AI model (e.g., GPT-3).

[1033] Example prompt:

[1034] Create a virtual travel-loving user who is a nature lover and travels frequently.

[1035] Server: Places the generated agent at an initial position in the virtual environment and allows the agent to move freely. Using the virtual environment engine, imports a 3D model and performs initial settings.

[1036] 4. Agent Activity Data Collection

[1037] Server: Activates the tracking system and tracks the agent's actions in real time, for example, recording the time, location, and interactions of the agent with other users in the virtual park.

[1038] Server: The agent's movements and activities are stored in a database as log data.

[1039] 5. Data Analysis and Visualization

[1040] Server: Analyzes the collected behavioral data to identify user behavioral patterns and preferences. Classifies the agent's behavior using clustering techniques and visually analyzes the data.

[1041] Server: Displays the analysis results using visualization tools (e.g., Tableau or Power BI), allowing companies to easily understand and utilize the data.

[1042] This invention enables comprehensive analysis of users' online and virtual behavioral data, enabling companies to efficiently utilize detailed data. This system contributes to the creation of new value and the improvement of services.

[1043] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1044] Step 1: Data collection

[1045] Input: Credentials for users to log in to the information sharing system

[1046] What happens: A user launches the application and clicks the "Log in with Facebook" button. The user enters their login information in the OAuth authentication screen that appears and grants the application the necessary data access permissions.

[1047] Processing: An OAuth token is obtained based on the authentication information, and the server uses the information sharing system's API to obtain data such as the user's posts, comments, images, and like history.

[1048] Output: Retrieved user data

[1049] Step 2: Data Preprocessing

[1050] Input: Acquired user data (text, images, etc.)

[1051] Specific operations: The server removes unnecessary HTML tags and special characters from text data, performs preprocessing such as using Python regular expressions, and standardizes the format and size of image data.

[1052] Processing: Clean and normalize the data.

[1053] Output: Preprocessed user data

[1054] Step 3: Data analysis

[1055] Input: Preprocessed user data (text, images, etc.)

[1056] How it works: The server uses natural language processing algorithms (e.g., Python's NLTK library) to extract keywords, emotions, and tones from text data. The server uses image analysis techniques (e.g., OpenCV) to recognize objects and scenes in images.

[1057] Processing: Extract keywords and emotions from text data, and recognize objects and scenes from image data.

[1058] Output: User characteristics as analysis results

[1059] Step 4: Creating a User Agent

[1060] Input: User characteristics

[1061] Specific operation: The server creates a prompt for the generated AI model (e.g., GPT-3) to input the user's characteristic information and inputs it into the model. The specific prompt is, "Please generate a user who loves to travel in the virtual space. This user is a nature lover and travels frequently."

[1062] Processing: A generative AI model generates a user representative in a virtual space based on input prompts.

[1063] Output: A user representative generated in the virtual space

[1064] Step 5: Collecting data on agent activities

[1065] Input: A user's representative generated in the virtual space

[1066] Specific operation: The server starts a tracking system to track the agent's actions in real time, recording the agent's location, movement route, and interactions with other agents.

[1067] Processing: Collecting agent behavior data and storing it in a database.

[1068] Output: Detailed agent behavior data

[1069] Step 6: Data analysis and visualization

[1070] Input: Detailed agent behavior data

[1071] Specific operations: Analyze the behavioral data collected by the server to identify behavioral patterns and preferences. Classify the behavioral data using clustering techniques. Display the analysis results using a visualization tool (e.g., Tableau or Power BI).

[1072] Processing: Analysis and visualization of behavioral data

[1073] Output: Analyzed behavioral patterns and preferences, visualized data

[1074] In this way, by having the user and server perform specific actions at each step, the system can perform detailed analysis of user behavior data and provide data that companies can use to create new value.

[1075] (Application example 1)

[1076] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1077] Conventional virtual user avatar systems have the problem that they do not adequately provide personalized product recommendations based on the user's interests. In addition, the collection and analysis of user behavior data in virtual space is insufficient, making it difficult for companies to obtain information that is useful for marketing strategies and new product development.

[1078] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1079] In this invention, the server includes means for allowing a user to link with a social networking service, means for acquiring user data from the social networking service, means for analyzing the user data and creating a user profile, means for generating an avatar of the user in a virtual space based on the profile, means for collecting behavioral data of the user's avatar in the virtual space, means for analyzing the collected behavioral data, means for visualizing the analysis results, and means for making product suggestions to the user in a virtual store based on the user's profile. This enables personalized product suggestions based on the user's interests and behavioral patterns, and allows companies to obtain detailed data for efficient and effective marketing strategies and new product development.

[1080] A "social networking service" is an online platform that enables users to connect, interact, and share information with other users over the Internet.

[1081] "User Data" refers collectively to information generated, shared, or engaged with by users on social networking services, including posts, comments, images, and like history.

[1082] A "virtual space" is an artificial three-dimensional space generated by a computer, in which users can interact through avatars.

[1083] An "alter ego" is an avatar generated in a virtual space based on a user's profile, and is a character that acts in the virtual space reflecting the user's actions, interests, and concerns.

[1084] A "virtual store" is a virtual store operated within a virtual space, where users can view and purchase virtual products.

[1085] "Personalized product suggestions" refers to the suggestion of individually customized products and services based on each user's interests and behavioral patterns.

[1086] "Natural language processing algorithms" are computational techniques for understanding, analyzing, and generating human language, and are used to extract meaning and sentiment from text data.

[1087] "Visualization" refers to the visual representation of data, making the results of data analysis intuitively understandable through dashboards, graphs, charts, etc.

[1088] This invention is a system that allows users to link with social networking services (SNS), analyzes user data acquired from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[1089] This system is realized with the following configuration.

[1090] Data collection

[1091] The user connects to their SNS account using their device. They enter their SNS account authentication information and grant access to the service. The server then calls the SNS API based on the user's authentication information to retrieve data such as the user's posts, comments, images, and like history.

[1092] Data analysis

[1093] The server preprocesses the acquired social media data, for example by cleaning the text and normalizing the images. It then uses natural language processing algorithms to extract keywords, emotions, and tones from the text data, and image analysis techniques to recognize objects and scenes in the images. Based on these analysis results, the server generates a user profile.

[1094] User avatar generation

[1095] The server inputs the user's profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space. The generated avatar is placed in an initial position in the virtual space and can move freely.

[1096] Collection of lifestyle data

[1097] The server starts a tracking system and tracks the actions of the avatar in real time, recording the avatar's movements and activities as log data and saving it in a database.

[1098] Data analysis and visualization

[1099] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, making it easy for companies to understand and utilize the data.

[1100] Product proposals in virtual stores

[1101] In a virtual store in a virtual space, a user's avatar interacts with other avatars, and personalized product suggestions are made based on their behavioral data, allowing for product suggestions based on the user's interests and behavioral patterns.

[1102] The main hardware required to realize this system is a smartphone, and the following software is used:

[1103] Platform: Android / iOS

[1104] Language: Java (Android), Swift (iOS)

[1105] API: Social networking service API (e.g. Facebook Graph API)

[1106] Libraries: Gson (JSON parsing), OkHttp3 (HTTP communication)

[1107] Generative AI model: Used to generate avatars based on user profiles

[1108] As a concrete example, the following prompt sentence can be used:

[1109] Example prompt sentence:

[1110] Generate an active user avatar in a virtual space based on the user's social media data (interests: travel, activity: hiking). Collect simulation data of the avatar searching for travel-related goods and activities in a virtual store. Display this data on a visualization dashboard, allowing travel-related companies to use it to propose new products.

[1111] This will enable companies to gain a detailed understanding of users' preferences and behavioral patterns, enabling them to propose more personalized products.

[1112] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1113] Step 1:

[1114] A user connects to a social networking account using a device. The user launches the application, enters their social networking account authentication information, and grants access to the service. At this time, the device sends the authentication information to the server and receives an authentication token to call the SNS API. The output is an authentication token.

[1115] Step 2:

[1116] The server uses the authentication token to call the SNS API and obtain data such as the user's posts, comments, images, and like history. This data is obtained in JSON format and temporarily stored on the server. The input is the authentication token, and the output is the user's SNS data.

[1117] Step 3:

[1118] The server preprocesses the acquired social media data. Specifically, it cleans the text (removes unnecessary tags and special characters) and normalizes the images. It uses natural language processing algorithms to extract keywords, emotions, and tones from the text data. It also uses image analysis techniques to recognize objects and scenes in the images. The input is the social media data, and the output is the analysis results.

[1119] Step 4:

[1120] The server generates a user profile based on the analysis results. This profile contains detailed information about the user, such as their interests and behavioral patterns. The profile is stored in a database. The input is the analysis results, and the output is the user profile.

[1121] Step 5:

[1122] The server inputs the user's profile information into the generative AI model and generates the user's avatar in the virtual space. The generated avatar is placed at an initial position in the virtual space and set up so that it can move freely. The input is the user profile, and the output is the user's avatar.

[1123] Step 6:

[1124] The server starts the tracking system and tracks the avatar's actions in real time. The avatar's movements and activities are recorded as log data and stored in a database. The input is the user's avatar, and the output is the behavior log.

[1125] Step 7:

[1126] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, allowing companies to easily understand and utilize the data. The input is the behavior log, and the output is the visualized analysis results.

[1127] Step 8:

[1128] In a virtual store in a virtual space, the user's avatar interacts with other avatars. The server uses this behavioral data to make personalized product suggestions. It identifies, highlights, and suggests products and services that the user may be interested in. The input is the behavioral log and user profile, and the output is product suggestions.

[1129] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1130] This invention is a system that allows users to link with social networking services (SNS), analyzes SNS data to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, by combining it with an emotion engine, we provide a platform that recognizes and analyzes the user's emotions and reflects that emotional information in the profile and behavioral data.

[1131] Program processing overview

[1132] 1. Data Collection

[1133] On the device: The user launches the application and clicks the button to link their SNS account. The user enters their login information for the SNS account and grants the app permission to access their data.

[1134] Server: Sends a request to the SNS API using the authentication token to retrieve user data (posts, comments, images, likes, etc.).

[1135] 2. Data Analysis

[1136] Server: Preprocesses the collected SNS data, cleaning text data and normalizing image data.

[1137] Server: Uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from text data. It also performs image analysis to extract features to identify user interests.

[1138] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[1139] 3. User avatar generation

[1140] Server: Enters profile information into the generative AI model and generates an avatar in the virtual space that reflects the user's characteristics and emotional state.

[1141] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[1142] 4. Collection of lifestyle data

[1143] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[1144] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[1145] 5. Data Analysis and Visualization

[1146] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns and preferences, as well as emotional information.

[1147] Server: Visually displays the analysis results using visualization tools, making it easy for companies to understand and utilize the data.

[1148] Specific examples

[1149] Examples of data collection

[1150] Device: The user launches the application and clicks the button to link their Facebook account.

[1151] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[1152] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[1153] Specific examples of data analysis

[1154] Server: Preprocesses the acquired data and removes unnecessary tags and special characters from the text data.

[1155] Server: Uses natural language processing models and sentiment engines to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[1156] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[1157] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[1158] Example of user avatar creation

[1159] Server: Based on the analysis results, a user profile is created with characteristics and emotional information such as "travel lover" or "nature lover."

[1160] Server: This profile is input into a generative AI model to generate an avatar of the user in a virtual space. The avatar can then behave in a way that reflects the user's emotional state.

[1161] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[1162] Examples of collecting lifestyle data

[1163] Server: Records the actions of the avatar in real time, storing logs of the places visited and the activities participated in. The actions of the avatar also include emotional information.

[1164] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail, analyzes the user's interests and behavioral patterns, and also records emotional information.

[1165] Specific examples of data utilization

[1166] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[1167] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[1168] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services. The addition of an emotion engine makes it possible to collect detailed data that reflects user emotions, resulting in more accurate analysis results.

[1169] The processing flow will be explained below.

[1170] Step 1:

[1171] Device: The user launches the application and clicks the button to link their social media account.

[1172] Step 2:

[1173] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[1174] Step 3:

[1175] Device: When the user authorizes the connection, an authentication token is generated.

[1176] Step 4:

[1177] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[1178] Step 5:

[1179] Server: Saves the acquired user data in a database.

[1180] Step 6:

[1181] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[1182] Step 7:

[1183] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[1184] Step 8:

[1185] Server: Analyzes and recognizes the user's emotional information using the emotion engine.

[1186] Step 9:

[1187] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[1188] Step 10:

[1189] Server: Creates a user profile based on all analysis results, including emotional information.

[1190] Step 11:

[1191] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[1192] Step 12:

[1193] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[1194] Step 13:

[1195] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[1196] Step 14:

[1197] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[1198] Step 15:

[1199] Server: Analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences, including emotional information.

[1200] Step 16:

[1201] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[1202] Step 17:

[1203] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[1204] This allows for the creation of a detailed profile from a user's social media data, the generation of an avatar based on that profile in a virtual space, and tracking of their behavior. The addition of an emotion engine makes it possible to collect and analyze detailed data that reflects user emotions, making it an extremely useful source of information for companies.

[1205] Example 2

[1206] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1207] In recent years, the number of users of social networking services has increased, leading to a diversification of the analysis and utilization of user data. However, conventional systems have had difficulty generating detailed profiles of users' emotions and behaviors, and tracking and analyzing their behavior in virtual spaces in real time. Furthermore, there are limited means for visually displaying the analysis results and for companies to effectively utilize them. Therefore, there is a need for a new system that can provide advanced analysis results, generate detailed profiles based on users' emotions and behaviors, and effectively utilize them in business strategies.

[1208] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1209] In this invention, the server includes a means for a user to link with a communication network service, a means for acquiring user data from the communication network service, a means for preprocessing the acquired user data, a means for analyzing the user data and creating a user profile, a means for generating an avatar of the user in a virtual environment based on the profile, a means for collecting behavioral data of the user's avatar in the virtual environment, a means for analyzing the collected behavioral data, and a means for visualizing the analysis results. This makes it possible to generate a detailed user profile and track and analyze behavior in a virtual space, thereby providing a system that can effectively utilize the collected data.

[1210] "User" refers to any individual or corporation that uses this system.

[1211] "Communication network services" is a general term for services that allow users to exchange information via the Internet.

[1212] "User Data" refers to all information generated by users of communication network services, including posts, comments, images, like history, etc.

[1213] "Preprocessing" refers to the process of formatting and cleaning up data before analysis, and includes removing unnecessary tags and special characters, normalizing image data, etc.

[1214] "Language analysis algorithm" refers to an algorithm that uses natural language processing technology to extract useful information (keywords, emotions, tone, etc.) from text data.

[1215] A "profile" refers to information that is compiled by analyzing user data and summarizing the user's interests, concerns, emotional state, etc.

[1216] A "virtual environment" is a space generated by computer simulation in which the user's avatar operates.

[1217] An "alter ego" refers to a character in a virtual environment that reflects the user's characteristics and emotional state.

[1218] "Behavioral data" refers to recorded information such as the movements, activities, and interactions of an avatar within a virtual environment.

[1219] "Visualization" refers to displaying analytical results in a visual form, such as a graph or chart, making the data easier to understand.

[1220] The present invention provides a system in which a user interacts with a communication network service, analyzes acquired user data, creates a detailed profile of the user, and generates an avatar of the user in a virtual environment based on the profile. Specific embodiments of this system are described below.

[1221] Data collection

[1222] User: The user launches the dedicated application and clicks a button to connect to a communication network service (e.g., SNS). On the displayed authentication screen, the user enters login information and grants the application permission to access data.

[1223] As a specific example, a user clicks the SNS link button and enters login information on the SNS's OAuth authentication page.

[1224] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data (posts, comments, images, likes, etc.). The server accesses the appropriate API endpoint and collects the user data.

[1225] As a concrete example, the server calls a social networking API to retrieve the user's latest posting data and like history.

[1226] Data analysis

[1227] Server: Preprocesses the acquired user data. Text data is cleaned of unnecessary tags and special characters, and image data is normalized.

[1228] As a specific example, the process involves removing HTML tags contained in text data and deleting emojis and special characters.

[1229] Server: Analyzes text data for keywords, sentiment, and tone using natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER), as well as image analysis (e.g., YOLOv3) to identify user interests.

[1230] As a specific example, a natural language processing model is used to extract frequently used phrases and word tones, and YOLOv3 is used to perform object recognition on image data.

[1231] Server: Using these analysis results, a user profile is generated. The profile includes the user's interests and emotional information. The generated profile is stored in a database.

[1232] As a specific example, the analysis results are compiled in a profile data format and written to a database.

[1233] User avatar generation

[1234] Server: Using the generated profile information as input, a generative AI model (e.g., GPT-3) is used to generate a virtual avatar for the user, which reflects the user's characteristics and emotional state.

[1235] As a specific example, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[1236] Server: The created avatar is placed at the initial position in the virtual environment. The avatar is set to be able to move freely.

[1237] As a specific example, using a virtual environment engine (e.g., Unity), the generated avatar is placed at a specific position in the virtual space and begins to act as an NPC (non-player character).

[1238] Collection of lifestyle data

[1239] Server: The tracking system is activated and the behavior of the avatar in the virtual environment is tracked in real time. The avatar's behavior also reflects emotional information.

[1240] As a specific example, logs of the avatar's movements, communications, and activities are collected in real time and recorded in a database.

[1241] Server: Collects and stores the behavioral data of the avatars in a database. The collected data includes detailed activity logs and emotional states.

[1242] As a specific example, activities performed by an avatar (e.g., visits to a virtual park and interactions with other avatars) are stored as log data.

[1243] Data analysis and visualization

[1244] Server: Analyzes collected behavioral data to identify detailed behavioral patterns, preferences, and emotional states of users.

[1245] As a specific example, the collected data is analyzed using data analysis tools (e.g., Python's Pandas) to identify behavioral patterns and emotional fluctuations.

[1246] Server: The analysis results are visually displayed using a visualization tool (e.g., Tableau), allowing companies to easily understand them and utilize them in their business strategies.

[1247] As a specific example, the analysis results can be displayed on a dashboard as graphs and charts, and can be viewed by companies.

[1248] Here's an example prompt: "Analyze the following social media data (posts, likes, and comments) and generate a user profile. Then, create a virtual avatar of the user based on this profile, track the avatar's behavior, and visualize the data."

[1249] This system allows the creation of detailed user profiles, real-time tracking and analysis of virtual avatar behavior, and the provision of a system that allows companies to effectively utilize the data. Furthermore, the introduction of an emotion engine enables advanced data analysis that reflects the user's emotional state.

[1250] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1251] Step 1: Data collection

[1252] User: The user launches a dedicated application and clicks a button to connect to the communication network service. They enter their login information on the displayed authentication screen and grant the application permission to access data. The input is the user's login information, and the output is an authentication token.

[1253] Specifically, the user clicks the SNS link button and enters their login information on the SNS's OAuth authentication page. This operation obtains an authentication token from the SNS.

[1254] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data. The input is the authentication token and the API request, and the output is user data (posts, comments, images, likes, etc.).

[1255] Specifically, the server calls the SNS API to retrieve the user's latest posts and likes. This data is used in the subsequent analysis steps.

[1256] Step 2: Data Preprocessing

[1257] Server: Preprocesses the acquired user data. The input is raw data and the output is preprocessed data. Data cleaning involves removing unnecessary tags and special characters from text data and normalizing image data.

[1258] Specifically, the system strips out HTML tags from the text data, removes emojis and special characters, and converts the image data into a format suitable for analysis.

[1259] Step 3: Data analysis

[1260] Server: Analyzes the preprocessed data. The input is the preprocessed data, and the output is the analysis results (keywords, sentiment, tone, etc.). It uses natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER) to extract information from text data. It also performs image analysis (e.g., YOLOv3).

[1261] Specifically, the BERT model is applied to text to extract keywords and perform sentiment analysis, and YOLOv3 is used to perform object recognition on image data.

[1262] Step 4: Generate a profile

[1263] Server: Generates a user profile based on the analysis results. The input is the analysis results, and the output is the user profile. The profile includes the user's interests, concerns, and emotional information.

[1264] Specifically, the extracted keywords and emotional information are compiled into profile data format and stored in a database.

[1265] Step 5: Creating a user avatar

[1266] Server: The generated profile information is input into a generative AI model (e.g., GPT-3) to generate the user's avatar in the virtual environment. The input is the user profile, and the output is the user's avatar (virtual character). The avatar reflects the user's characteristics and emotional state.

[1267] Specifically, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[1268] Step 6: Place your clones

[1269] Server: The generated avatar is placed at the initial position in the virtual environment. The input is the generated user avatar, and the output is the avatar in the virtual environment. The avatar is set to be able to act freely.

[1270] Specifically, the generated avatar is placed at a specific position in the virtual space using a virtual environment engine (e.g., Unity). This avatar is designed to interact with other avatars and elements.

[1271] Step 7: Collecting lifestyle data

[1272] Server: Activates the tracking system and tracks the behavior of the avatar in the virtual environment in real time. The input is the activity data of the avatar in the virtual environment, and the output is the behavior log. The behavior of the avatar also includes emotional information.

[1273] Specifically, the system collects logs of the avatar's movements, communications, and activities in real time and records them in a database.

[1274] Step 8: Data analysis and visualization

[1275] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns, preferences, and emotional states. The input is behavioral data, and the output is the analysis results. Data analysis tools (e.g., Python's Pandas) are used to analyze the collected data.

[1276] Specifically, the collected data is analyzed using data analysis tools to identify behavioral patterns and emotional fluctuations.

[1277] Server: Visually displays the analysis results using a visualization tool (e.g., Tableau). The input is the analysis results, and the output is visualized data (graphs, charts, etc.).

[1278] Specifically, the analysis results are displayed on a dashboard as graphs and charts, allowing companies to easily understand the data and use it in their business strategies.

[1279] In this way, the present invention provides a system that generates detailed profiles of users, tracks and analyzes the behavior of their virtual avatars in real time, and allows companies to effectively utilize the data.

[1280] (Application example 2)

[1281] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1282] Conventional user behavior analysis systems using virtual spaces or social networking services (SNS) have had difficulty providing personalized content that effectively reflects users' interests and emotions. Furthermore, there has been a lack of systems that can collect users' emotional states in real time and use that data to recommend future content. This has limited the improvement of user experience.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1284] In this invention, the server includes: means for a user to link with a social networking service; means for acquiring user data from the social networking service; means for analyzing the user data and creating a user profile; means for generating an avatar of the user in a virtual space based on the profile; means for collecting behavioral data of the user's avatar in the virtual space; means for analyzing the collected behavioral data; means for visualizing the analysis results; means for inserting the generated avatar of the user into video content; means for recommending video content based on the user profile; means for collecting emotional data of the user while watching a video; and means for analyzing the collected emotional data and reflecting it in the next video content recommendation. This makes it possible to provide personalized content that effectively reflects the user's interests and emotions and to improve the user experience.

[1285] A "social networking service" is an online platform that allows users to communicate with each other and share information via the Internet.

[1286] "User Data" refers collectively to user posts, comments, likes, images, and their history information collected on social networking services.

[1287] A "profile" is a set of information generated based on user data, including characteristics such as a user's interests, concerns, behavioral patterns, and emotional state.

[1288] A "virtual space" is a digital world that is generated using computer graphics and other techniques, and in which users can participate interactively.

[1289] A "user's avatar" is a digital avatar that imitates the user and is generated in a virtual space based on the user's profile.

[1290] "Behavioral data" is a record of the actions, movements, and emotional state of the user's avatar in the virtual space.

[1291] "Analysis results" is a general term for insights and information obtained after analyzing collected user data and behavioral data.

[1292] "Visualization" is a technique for visually expressing analytical results in a form that is easy to understand.

[1293] "Video content" is a general term for video and audio distributed over the Internet.

[1294] "Emotional data" is information about a user's emotional state that is analyzed from the user's posts and behavior.

[1295] The following describes an embodiment of the present invention. This system collects and analyzes a user's social networking service (SNS) data to create a user profile and provides personalized video content based on the results. Specifically, it inserts an avatar created using the user data into the video content, and collects emotional data from the user while watching the video to use in recommending the next content.

[1296] The server uses the following hardware and software:

[1297] Hardware:

[1298] Smartphones (including iPhones and Android devices)

[1299] Servers (high-performance computers for analysis and data processing)

[1300] software:

[1301] Python (programming language)

[1302] TensorFlow (a library for implementing deep learning models)

[1303] NLTK (Natural Language Processing Toolkit)

[1304] Django (web framework)

[1305] RESTful API (interface for exchanging data)

[1306] After the user installs the application on their smartphone, the system operates as follows.

[1307] 1. Data Collection:

[1308] A user launches the application and links their social media account. This linking sends data such as the user's posts, comments, and like history to the server using the social media API. For example, a user opens the application, clicks the button to link their Instagram account, and enters their login information on the OAuth authentication screen. This operation retrieves the user's photo posts and comments via the Instagram API.

[1309] 2. Data Analysis:

[1310] The server preprocesses the collected social media data, cleaning the text data and normalizing the image data, then uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from the text data and identify user interests through image analysis.

[1311] 3. User avatar generation:

[1312] Using a generative AI model, the system generates a virtual avatar of the user based on the analyzed profile data, and the avatar can behave in a way that reflects the user's emotions and interests.

[1313] 4. Personalize your video content:

[1314] The server recommends optimal video content based on the user profile. Furthermore, the server inserts the generated avatar into the video and acts as a guide based on the user's interests. For example, if the user likes "travel," the server recommends videos related to travel, in which the user's avatar guides them around tourist spots.

[1315] 5. Collecting Emotional Data:

[1316] While the user is watching the video, the server collects emotional data in real time, which is recorded as post-viewing feedback and used to recommend content for the next time.

[1317] Below are some examples of prompts to use when analyzing collected data:

[1318] Example prompt sentence:

[1319] Extract the following information for a user's Instagram posts:

[1320] keyword

[1321] Emotional state

[1322] Frequent phrases for each keyword

[1323] Based on these prompts, useful information is extracted from the user's posts and used to personalize the video content. The app also takes into account the user's emotional state to recommend the most suitable videos, and the user's avatar acts as a guide within the video, providing a more immersive experience.

[1324] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1325] Program processing flow and detailed explanation of each step

[1326] Step 1:

[1327] Data collection

[1328] Input: The user launches the application and links their social media account.

[1329] Operation: After the user clicks the SNS link button, the device displays the OAuth authentication screen. The user enters their login information and data access permissions.

[1330] Output: SNS authentication token and permissions to access user data are obtained.

[1331] How it works: The server uses the obtained authentication token to send a request to the SNS API to retrieve data such as the user's posts, comments, and likes.

[1332] Step 2:

[1333] Data Preprocessing

[1334] Input: Retrieved social media data.

[1335] How it works: The server cleans the collected data by removing unnecessary tags and special characters, and also normalizes the image data.

[1336] Output: Cleaned text data and normalized image data.

[1337] What it does: Pass this clean data on to the next analysis step.

[1338] Step 3:

[1339] Data analysis

[1340] Input: Cleaned text data and normalized image data.

[1341] How it works: The server uses natural language processing algorithms (e.g., NLTK) and emotion engines to extract keywords, emotional states, and tone from text data. It also performs image analysis to extract features that identify user interests.

[1342] Output: User's keyword list, emotional state and areas of interest.

[1343] How it works: Generates a user profile based on the analysis results.

[1344] Step 4:

[1345] User avatar generation

[1346] Input: The generated user profile.

[1347] How it works: The server uses a generative AI model to generate a virtual avatar for the user based on the user's profile, which reflects the avatar's characteristics and emotional state.

[1348] Output: A virtual avatar of the user.

[1349] Movement: Set the avatar to be able to move freely within the virtual space.

[1350] Step 5:

[1351] Personalized video content

[1352] Input: User profile and virtual self.

[1353] How it works: The server recommends the most suitable video content based on the user profile and inserts the user's avatar into the video. The avatar appears as a navigator in the video, guiding the user through the content.

[1354] Output: Personalized video content.

[1355] What happens: Video is streamed to your device.

[1356] Step 6:

[1357] Collecting Emotional Data

[1358] Input: The video content that the user watches.

[1359] How it works: The device collects the user's emotions in real time while watching, providing post-viewing feedback.

[1360] Output: Real-time collected emotion data.

[1361] Operation: The server analyzes this emotional data and reflects it in the next video content recommendation.

[1362] Step 7:

[1363] Using Feedback

[1364] Input: Collected real-time emotion data.

[1365] How it works: The server analyzes the collected emotional data and reflects it in the next video content recommendation. Specifically, it adjusts the recommendation algorithm based on changes in the user's emotional state and reactions to specific content.

[1366] Output: Upcoming video content recommendation list.

[1367] What it does: Provides users with a list of upcoming video content recommendations.

[1368] Through these steps, the system provides personalized content that effectively reflects the user's interests and emotions.

[1369] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1370] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1371] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1372] [Fourth embodiment]

[1373] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1374] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1375] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1376] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1377] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1378] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1379] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1380] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1381] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1382] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1383] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1384] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1385] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1386] This invention is a system that allows users to connect with social networking services (SNS), analyzes user data obtained from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[1387] Program processing overview

[1388] 1. Data Collection

[1389] Device: The user links their SNS account to the application. At this time, the user enters their SNS account authentication information and grants access rights to the service.

[1390] Server: Calls the SNS API based on the user's authentication information and obtains data such as the user's posts, comments, images, and like history.

[1391] 2. Data Analysis

[1392] Server: Preprocesses the acquired SNS data, for example by cleaning text and normalizing images.

[1393] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data. Also performs image analysis to extract features to identify user interests.

[1394] Server: Generates a user profile based on these analysis results.

[1395] 3. User avatar generation

[1396] Server: Enters profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space.

[1397] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[1398] 4. Collection of lifestyle data

[1399] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[1400] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[1401] 5. Data Analysis and Visualization

[1402] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[1403] Server: Visually displays the analysis results using visualization tools, allowing companies to easily understand and utilize the data.

[1404] Specific examples

[1405] Examples of data collection

[1406] Device: The user launches the application and clicks the button to link their Facebook account.

[1407] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[1408] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[1409] Specific examples of data analysis

[1410] Server: In order to analyze the collected data, unnecessary tags and special characters are removed from the text data.

[1411] Server: Uses natural language processing models to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[1412] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[1413] Example of user avatar creation

[1414] Server: Based on the analysis results, a profile of the user is created with characteristics such as "travel lover" or "nature lover."

[1415] Server: This profile is input into the generative AI model, and an avatar of the user is generated in the virtual space.

[1416] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[1417] Examples of collecting lifestyle data

[1418] Server: Records the actions of your avatar in real time, storing logs of the places you visit and the activities you participate in.

[1419] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail and analyzes the user's interests and behavioral patterns.

[1420] Specific examples of data utilization

[1421] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[1422] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[1423] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services.

[1424] The processing flow will be explained below.

[1425] Step 1:

[1426] Device: The user launches the application and clicks the button to link their social media account.

[1427] Step 2:

[1428] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[1429] Step 3:

[1430] Device: When the user authorizes the connection, an authentication token is generated.

[1431] Step 4:

[1432] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[1433] Step 5:

[1434] Server: Saves the acquired user data in a database.

[1435] Step 6:

[1436] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[1437] Step 7:

[1438] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[1439] Step 8:

[1440] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[1441] Step 9:

[1442] Server: Creates a user profile based on all analysis results.

[1443] Step 10:

[1444] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[1445] Step 11:

[1446] Server: Places the generated avatar at its initial position in the virtual space.

[1447] Step 12:

[1448] Server: Activates the tracking system and tracks the actions of the avatar in real time.

[1449] Step 13:

[1450] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[1451] Step 14:

[1452] Server: Analyzes collected behavioral data to identify detailed user behavior patterns and preferences.

[1453] Step 15:

[1454] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[1455] Step 16:

[1456] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[1457] This allows companies to create detailed profiles of users from their social media data, generate virtual avatars based on those profiles, and track their behavior. The collected data is a very useful source of information for companies.

[1458] Example 1

[1459] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1460] In today's information society, companies and organizations aim to create new value and improve their services by analyzing and utilizing detailed user behavior data. However, with current technology, it has been difficult to efficiently analyze and visualize detailed data, including not only users' online behavior but also their behavioral patterns in virtual environments. Furthermore, technologies for automatically generating and tracking behavior in virtual environments directly from user data have been limited, restricting companies' multifaceted use of data.

[1461] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1462] In this invention, the server includes a means for a user to link with the information sharing system, a means for acquiring user data from the information sharing system, a means for analyzing the user data and creating user characteristics, a means for generating a user representative in a virtual environment based on the characteristics, a means for collecting activity data of the user representative in the virtual environment, a means for analyzing the collected activity data, and a means for visualizing the analysis results. This enables comprehensive analysis of user behavior data online and in the virtual environment, enabling companies to efficiently utilize detailed data.

[1463] "User" refers to an individual or organization that uses the system.

[1464] "Information sharing system" refers to online platforms that provide user data, such as social networking services.

[1465] "User Data" refers to digital information obtained from the information sharing system, such as user posts, comments, images, and like history.

[1466] "Analysis" refers to the process of processing acquired user data and extracting useful information and trends.

[1467] "Traits" refer to information about a user's interests and behavioral patterns generated from analyzed data.

[1468] A "virtual environment" refers to a digital space simulated by a computer.

[1469] "Proxy" refers to a digital avatar that is generated to simulate the user's actions within a virtual environment.

[1470] "Activity Data" refers to a record of actions taken by an agent within a virtual environment.

[1471] "Visualization" refers to the process of displaying analytical results in the form of diagrams, charts, etc., to make the data easier to understand.

[1472] MODE FOR CARRYING OUT THE INVENTION

[1473] The present invention is a system that allows users to connect to an information sharing system, analyzes the user data acquired from the system, creates user characteristics, and generates a user's agent in a virtual environment. It also provides a platform that collects and analyzes the agent's activity data in the virtual environment and enables companies to utilize detailed data.

[1474] 1. Data collection and collaboration

[1475] Terminal: To access the information sharing system, a user launches an application on the terminal and clicks, for example, a "Log in with Facebook" button. The user enters their login information and grants the necessary data access permissions to link with the information sharing system.

[1476] Server: Using the obtained authentication token, the server calls the API of the information sharing system and obtains data such as the user's posts, comments, images, and like history. For example, the server obtains user data using the Facebook API.

[1477] 2. Data Analysis

[1478] Server: Preprocesses the collected data. For example, it uses Python regular expressions to remove unnecessary HTML tags and special characters from text data. Similarly, it standardizes the format and size of image data.

[1479] Server: Uses natural language processing algorithms to extract keywords, emotions, and tones from text data. For example, it uses Python's NLTK library to classify positive and negative emotions. It also performs image analysis using OpenCV to recognize objects and scenes in images.

[1480] Server: Based on the results of these analyses, the server generates user characteristics, including user interests and behavioral patterns.

[1481] 3. Creating a User Agent

[1482] Server: Input user characteristics into the generative AI model and generate a representative with the user's characteristics in the virtual environment. For example, input the prompt "Generate a user who likes to travel in the virtual space" into the generative AI model (e.g., GPT-3).

[1483] Example prompt:

[1484] Create a virtual travel-loving user who is a nature lover and travels frequently.

[1485] Server: Places the generated agent at an initial position in the virtual environment and allows the agent to move freely. Using the virtual environment engine, imports a 3D model and performs initial settings.

[1486] 4. Agent Activity Data Collection

[1487] Server: Activates the tracking system and tracks the agent's actions in real time, for example, recording the time, location, and interactions of the agent with other users in the virtual park.

[1488] Server: The agent's movements and activities are stored in a database as log data.

[1489] 5. Data Analysis and Visualization

[1490] Server: Analyzes the collected behavioral data to identify user behavioral patterns and preferences. Classifies the agent's behavior using clustering techniques and visually analyzes the data.

[1491] Server: Displays the analysis results using visualization tools (e.g., Tableau or Power BI), allowing companies to easily understand and utilize the data.

[1492] This invention enables comprehensive analysis of users' online and virtual behavioral data, enabling companies to efficiently utilize detailed data. This system contributes to the creation of new value and the improvement of services.

[1493] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1494] Step 1: Data collection

[1495] Input: Credentials for users to log in to the information sharing system

[1496] What happens: A user launches the application and clicks the "Log in with Facebook" button. The user enters their login information in the OAuth authentication screen that appears and grants the application the necessary data access permissions.

[1497] Processing: An OAuth token is obtained based on the authentication information, and the server uses the information sharing system's API to obtain data such as the user's posts, comments, images, and like history.

[1498] Output: Retrieved user data

[1499] Step 2: Data Preprocessing

[1500] Input: Acquired user data (text, images, etc.)

[1501] Specific operations: The server removes unnecessary HTML tags and special characters from text data, performs preprocessing such as using Python regular expressions, and standardizes the format and size of image data.

[1502] Processing: Clean and normalize the data.

[1503] Output: Preprocessed user data

[1504] Step 3: Data analysis

[1505] Input: Preprocessed user data (text, images, etc.)

[1506] How it works: The server uses natural language processing algorithms (e.g., Python's NLTK library) to extract keywords, emotions, and tones from text data. The server uses image analysis techniques (e.g., OpenCV) to recognize objects and scenes in images.

[1507] Processing: Extract keywords and emotions from text data, and recognize objects and scenes from image data.

[1508] Output: User characteristics as analysis results

[1509] Step 4: Creating a User Agent

[1510] Input: User characteristics

[1511] Specific operation: The server creates a prompt for the generated AI model (e.g., GPT-3) to input the user's characteristic information and inputs it into the model. The specific prompt is, "Please generate a user who loves to travel in the virtual space. This user is a nature lover and travels frequently."

[1512] Processing: A generative AI model generates a user representative in a virtual space based on input prompts.

[1513] Output: A user representative generated in the virtual space

[1514] Step 5: Collecting data on agent activities

[1515] Input: A user's representative generated in the virtual space

[1516] Specific operation: The server starts a tracking system to track the agent's actions in real time, recording the agent's location, movement route, and interactions with other agents.

[1517] Processing: Collecting agent behavior data and storing it in a database.

[1518] Output: Detailed agent behavior data

[1519] Step 6: Data analysis and visualization

[1520] Input: Detailed agent behavior data

[1521] Specific operations: Analyze the behavioral data collected by the server to identify behavioral patterns and preferences. Classify the behavioral data using clustering techniques. Display the analysis results using a visualization tool (e.g., Tableau or Power BI).

[1522] Processing: Analysis and visualization of behavioral data

[1523] Output: Analyzed behavioral patterns and preferences, visualized data

[1524] In this way, by having the user and server perform specific actions at each step, the system can perform detailed analysis of user behavior data and provide data that companies can use to create new value.

[1525] (Application example 1)

[1526] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1527] Conventional virtual user avatar systems have the problem that they do not adequately provide personalized product recommendations based on the user's interests. In addition, the collection and analysis of user behavior data in virtual space is insufficient, making it difficult for companies to obtain information that is useful for marketing strategies and new product development.

[1528] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1529] In this invention, the server includes means for allowing a user to link with a social networking service, means for acquiring user data from the social networking service, means for analyzing the user data and creating a user profile, means for generating an avatar of the user in a virtual space based on the profile, means for collecting behavioral data of the user's avatar in the virtual space, means for analyzing the collected behavioral data, means for visualizing the analysis results, and means for making product suggestions to the user in a virtual store based on the user's profile. This enables personalized product suggestions based on the user's interests and behavioral patterns, and allows companies to obtain detailed data for efficient and effective marketing strategies and new product development.

[1530] A "social networking service" is an online platform that enables users to connect, interact, and share information with other users over the Internet.

[1531] "User Data" refers collectively to information generated, shared, or engaged with by users on social networking services, including posts, comments, images, and like history.

[1532] A "virtual space" is an artificial three-dimensional space generated by a computer, in which users can interact through avatars.

[1533] An "alter ego" is an avatar generated in a virtual space based on a user's profile, and is a character that acts in the virtual space reflecting the user's actions, interests, and concerns.

[1534] A "virtual store" is a virtual store operated within a virtual space, where users can view and purchase virtual products.

[1535] "Personalized product suggestions" refers to the suggestion of individually customized products and services based on each user's interests and behavioral patterns.

[1536] "Natural language processing algorithms" are computational techniques for understanding, analyzing, and generating human language, and are used to extract meaning and sentiment from text data.

[1537] "Visualization" refers to the visual representation of data, making the results of data analysis intuitively understandable through dashboards, graphs, charts, etc.

[1538] This invention is a system that allows users to link with social networking services (SNS), analyzes user data acquired from the SNS to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, it provides a platform that collects and analyzes behavioral data of the avatar in the virtual space, allowing companies to utilize detailed data.

[1539] This system is realized with the following configuration.

[1540] Data collection

[1541] The user connects to their SNS account using their device. They enter their SNS account authentication information and grant access to the service. The server then calls the SNS API based on the user's authentication information to retrieve data such as the user's posts, comments, images, and like history.

[1542] Data analysis

[1543] The server preprocesses the acquired social media data, for example by cleaning the text and normalizing the images. It then uses natural language processing algorithms to extract keywords, emotions, and tones from the text data, and image analysis techniques to recognize objects and scenes in the images. Based on these analysis results, the server generates a user profile.

[1544] User avatar generation

[1545] The server inputs the user's profile information into the generative AI model and generates an avatar with the user's characteristics in the virtual space. The generated avatar is placed in an initial position in the virtual space and can move freely.

[1546] Collection of lifestyle data

[1547] The server starts a tracking system and tracks the actions of the avatar in real time, recording the avatar's movements and activities as log data and saving it in a database.

[1548] Data analysis and visualization

[1549] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, making it easy for companies to understand and utilize the data.

[1550] Product proposals in virtual stores

[1551] In a virtual store in a virtual space, a user's avatar interacts with other avatars, and personalized product suggestions are made based on their behavioral data, allowing for product suggestions based on the user's interests and behavioral patterns.

[1552] The main hardware required to realize this system is a smartphone, and the following software is used:

[1553] Platform: Android / iOS

[1554] Language: Java (Android), Swift (iOS)

[1555] API: Social networking service API (e.g. Facebook Graph API)

[1556] Libraries: Gson (JSON parsing), OkHttp3 (HTTP communication)

[1557] Generative AI model: Used to generate avatars based on user profiles

[1558] As a concrete example, the following prompt sentence can be used:

[1559] Example prompt sentence:

[1560] Generate an active user avatar in a virtual space based on the user's social media data (interests: travel, activity: hiking). Collect simulation data of the avatar searching for travel-related goods and activities in a virtual store. Display this data on a visualization dashboard, allowing travel-related companies to use it to propose new products.

[1561] This will enable companies to gain a detailed understanding of users' preferences and behavioral patterns, enabling them to propose more personalized products.

[1562] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1563] Step 1:

[1564] A user connects to a social networking account using a device. The user launches the application, enters their social networking account authentication information, and grants access to the service. At this time, the device sends the authentication information to the server and receives an authentication token to call the SNS API. The output is an authentication token.

[1565] Step 2:

[1566] The server uses the authentication token to call the SNS API and obtain data such as the user's posts, comments, images, and like history. This data is obtained in JSON format and temporarily stored on the server. The input is the authentication token, and the output is the user's SNS data.

[1567] Step 3:

[1568] The server preprocesses the acquired social media data. Specifically, it cleans the text (removes unnecessary tags and special characters) and normalizes the images. It uses natural language processing algorithms to extract keywords, emotions, and tones from the text data. It also uses image analysis techniques to recognize objects and scenes in the images. The input is the social media data, and the output is the analysis results.

[1569] Step 4:

[1570] The server generates a user profile based on the analysis results. This profile contains detailed information about the user, such as their interests and behavioral patterns. The profile is stored in a database. The input is the analysis results, and the output is the user profile.

[1571] Step 5:

[1572] The server inputs the user's profile information into the generative AI model and generates the user's avatar in the virtual space. The generated avatar is placed at an initial position in the virtual space and set up so that it can move freely. The input is the user profile, and the output is the user's avatar.

[1573] Step 6:

[1574] The server starts the tracking system and tracks the avatar's actions in real time. The avatar's movements and activities are recorded as log data and stored in a database. The input is the user's avatar, and the output is the behavior log.

[1575] Step 7:

[1576] The server analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences. The analysis results are displayed visually using a visualization tool, allowing companies to easily understand and utilize the data. The input is the behavior log, and the output is the visualized analysis results.

[1577] Step 8:

[1578] In a virtual store in a virtual space, the user's avatar interacts with other avatars. The server uses this behavioral data to make personalized product suggestions. It identifies, highlights, and suggests products and services that the user may be interested in. The input is the behavioral log and user profile, and the output is product suggestions.

[1579] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1580] This invention is a system that allows users to link with social networking services (SNS), analyzes SNS data to create a user profile, and generates an avatar of the user in a virtual space based on that profile. Furthermore, by combining it with an emotion engine, we provide a platform that recognizes and analyzes the user's emotions and reflects that emotional information in the profile and behavioral data.

[1581] Program processing overview

[1582] 1. Data Collection

[1583] On the device: The user launches the application and clicks the button to link their SNS account. The user enters their login information for the SNS account and grants the app permission to access their data.

[1584] Server: Sends a request to the SNS API using the authentication token to retrieve user data (posts, comments, images, likes, etc.).

[1585] 2. Data Analysis

[1586] Server: Preprocesses the collected SNS data, cleaning text data and normalizing image data.

[1587] Server: Uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from text data. It also performs image analysis to extract features to identify user interests.

[1588] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[1589] 3. User avatar generation

[1590] Server: Enters profile information into the generative AI model and generates an avatar in the virtual space that reflects the user's characteristics and emotional state.

[1591] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[1592] 4. Collection of lifestyle data

[1593] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[1594] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[1595] 5. Data Analysis and Visualization

[1596] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns and preferences, as well as emotional information.

[1597] Server: Visually displays the analysis results using visualization tools, making it easy for companies to understand and utilize the data.

[1598] Specific examples

[1599] Examples of data collection

[1600] Device: The user launches the application and clicks the button to link their Facebook account.

[1601] On the device: The user enters their login information on the OAuth authentication screen that appears and grants the app permission to access data.

[1602] Server: Calls the Facebook API using the authentication token to retrieve data such as the user's posts, photos, and like history.

[1603] Specific examples of data analysis

[1604] Server: Preprocesses the acquired data and removes unnecessary tags and special characters from the text data.

[1605] Server: Uses natural language processing models and sentiment engines to analyze text data for keywords and sentiment, extracting frequently used phrases and verbal tones.

[1606] Server: Uses image analysis technology to recognize objects and scenes in images and identify themes and places that interest the user.

[1607] Server: Based on the results of these analyses, a user profile is generated, which also includes the user's emotional information.

[1608] Example of user avatar creation

[1609] Server: Based on the analysis results, a user profile is created with characteristics and emotional information such as "travel lover" or "nature lover."

[1610] Server: This profile is input into a generative AI model to generate an avatar of the user in a virtual space. The avatar can then behave in a way that reflects the user's emotional state.

[1611] Server: The avatar begins living in a natural environment within the virtual space and interacting with other virtual residents.

[1612] Examples of collecting lifestyle data

[1613] Server: Records the actions of the avatar in real time, storing logs of the places visited and the activities participated in. The actions of the avatar also include emotional information.

[1614] Server: For example, if an avatar visits a virtual nature park and interacts with other avatars there, the server records the data in detail, analyzes the user's interests and behavioral patterns, and also records emotional information.

[1615] Specific examples of data utilization

[1616] Server: Displays the collected data in a visualization dashboard, allowing companies to easily analyze it.

[1617] Businesses: Based on the visualized data, they can consider proposing new travel packages and services, and use it to design marketing campaigns to effectively reach target users.

[1618] In this way, the present invention enables companies to efficiently collect and use high-quality, detailed user data, thereby creating new value and improving services. The addition of an emotion engine makes it possible to collect detailed data that reflects user emotions, resulting in more accurate analysis results.

[1619] The processing flow will be explained below.

[1620] Step 1:

[1621] Device: The user launches the application and clicks the button to link their social media account.

[1622] Step 2:

[1623] Device: The OAuth authentication screen is displayed, and the user enters the login information for their SNS account on that screen.

[1624] Step 3:

[1625] Device: When the user authorizes the connection, an authentication token is generated.

[1626] Step 4:

[1627] Server: Using the authentication token, send a request to the social networking service's API to retrieve user data (posts, comments, images, likes, etc.).

[1628] Step 5:

[1629] Server: Saves the acquired user data in a database.

[1630] Step 6:

[1631] Server: Preprocessing is performed to analyze the social media data, specifically cleaning the text data and resizing and normalizing images.

[1632] Step 7:

[1633] Server: Uses natural language processing algorithms to extract keywords, sentiment, and tone from text data.

[1634] Step 8:

[1635] Server: Analyzes and recognizes the user's emotional information using the emotion engine.

[1636] Step 9:

[1637] Server: Using image analysis technology, it recognizes objects and scenes from image data and identifies the user's interests.

[1638] Step 10:

[1639] Server: Creates a user profile based on all analysis results, including emotional information.

[1640] Step 11:

[1641] Server: Enters profile information into the generative AI model and generates an avatar of the user in the virtual space.

[1642] Step 12:

[1643] Server: Places the generated avatar at its initial position in the virtual space, allowing the avatar to move freely.

[1644] Step 13:

[1645] Server: The tracking system is activated and the actions of the avatar are tracked in real time. The avatar's actions also reflect emotional information.

[1646] Step 14:

[1647] Server: Records the movements and activities of the avatar as log data and stores it in a database.

[1648] Step 15:

[1649] Server: Analyzes the collected behavioral data to identify detailed user behavioral patterns and preferences, including emotional information.

[1650] Step 16:

[1651] Server: Visually displays the analysis results using a visualization tool, allowing companies to easily view the data.

[1652] Step 17:

[1653] Companies: Use the collected and analyzed data to improve products and services, build new business models, etc.

[1654] This allows for the creation of a detailed profile from a user's social media data, the generation of an avatar based on that profile in a virtual space, and tracking of their behavior. The addition of an emotion engine makes it possible to collect and analyze detailed data that reflects user emotions, making it an extremely useful source of information for companies.

[1655] Example 2

[1656] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1657] In recent years, the number of users of social networking services has increased, leading to a diversification of the analysis and utilization of user data. However, conventional systems have had difficulty generating detailed profiles of users' emotions and behaviors, and tracking and analyzing their behavior in virtual spaces in real time. Furthermore, there are limited means for visually displaying the analysis results and for companies to effectively utilize them. Therefore, there is a need for a new system that can provide advanced analysis results, generate detailed profiles based on users' emotions and behaviors, and effectively utilize them in business strategies.

[1658] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1659] In this invention, the server includes a means for a user to link with a communication network service, a means for acquiring user data from the communication network service, a means for preprocessing the acquired user data, a means for analyzing the user data and creating a user profile, a means for generating an avatar of the user in a virtual environment based on the profile, a means for collecting behavioral data of the user's avatar in the virtual environment, a means for analyzing the collected behavioral data, and a means for visualizing the analysis results. This makes it possible to generate a detailed user profile and track and analyze behavior in a virtual space, thereby providing a system that can effectively utilize the collected data.

[1660] "User" refers to any individual or corporation that uses this system.

[1661] "Communication network services" is a general term for services that allow users to exchange information via the Internet.

[1662] "User Data" refers to all information generated by users of communication network services, including posts, comments, images, like history, etc.

[1663] "Preprocessing" refers to the process of formatting and cleaning up data before analysis, and includes removing unnecessary tags and special characters, normalizing image data, etc.

[1664] "Language analysis algorithm" refers to an algorithm that uses natural language processing technology to extract useful information (keywords, emotions, tone, etc.) from text data.

[1665] A "profile" refers to information that is compiled by analyzing user data and summarizing the user's interests, concerns, emotional state, etc.

[1666] A "virtual environment" is a space generated by computer simulation in which the user's avatar operates.

[1667] An "alter ego" refers to a character in a virtual environment that reflects the user's characteristics and emotional state.

[1668] "Behavioral data" refers to recorded information such as the movements, activities, and interactions of an avatar within a virtual environment.

[1669] "Visualization" refers to displaying analytical results in a visual form, such as a graph or chart, making the data easier to understand.

[1670] The present invention provides a system in which a user interacts with a communication network service, analyzes acquired user data, creates a detailed profile of the user, and generates an avatar of the user in a virtual environment based on the profile. Specific embodiments of this system are described below.

[1671] Data collection

[1672] User: The user launches the dedicated application and clicks a button to connect to a communication network service (e.g., SNS). On the displayed authentication screen, the user enters login information and grants the application permission to access data.

[1673] As a specific example, a user clicks the SNS link button and enters login information on the SNS's OAuth authentication page.

[1674] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data (posts, comments, images, likes, etc.). The server accesses the appropriate API endpoint and collects the user data.

[1675] As a concrete example, the server calls a social networking API to retrieve the user's latest posting data and like history.

[1676] Data analysis

[1677] Server: Preprocesses the acquired user data. Text data is cleaned of unnecessary tags and special characters, and image data is normalized.

[1678] As a specific example, the process involves removing HTML tags contained in text data and deleting emojis and special characters.

[1679] Server: Analyzes text data for keywords, sentiment, and tone using natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER), as well as image analysis (e.g., YOLOv3) to identify user interests.

[1680] As a specific example, a natural language processing model is used to extract frequently used phrases and word tones, and YOLOv3 is used to perform object recognition on image data.

[1681] Server: Using these analysis results, a user profile is generated. The profile includes the user's interests and emotional information. The generated profile is stored in a database.

[1682] As a specific example, the analysis results are compiled in a profile data format and written to a database.

[1683] User avatar generation

[1684] Server: Using the generated profile information as input, a generative AI model (e.g., GPT-3) is used to generate a virtual avatar for the user, which reflects the user's characteristics and emotional state.

[1685] As a specific example, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[1686] Server: The created avatar is placed at the initial position in the virtual environment. The avatar is set to be able to move freely.

[1687] As a specific example, using a virtual environment engine (e.g., Unity), the generated avatar is placed at a specific position in the virtual space and begins to act as an NPC (non-player character).

[1688] Collection of lifestyle data

[1689] Server: The tracking system is activated and the behavior of the avatar in the virtual environment is tracked in real time. The avatar's behavior also reflects emotional information.

[1690] As a specific example, logs of the avatar's movements, communications, and activities are collected in real time and recorded in a database.

[1691] Server: Collects and stores the behavioral data of the avatars in a database. The collected data includes detailed activity logs and emotional states.

[1692] As a specific example, activities performed by an avatar (e.g., visits to a virtual park and interactions with other avatars) are stored as log data.

[1693] Data analysis and visualization

[1694] Server: Analyzes collected behavioral data to identify detailed behavioral patterns, preferences, and emotional states of users.

[1695] As a specific example, the collected data is analyzed using data analysis tools (e.g., Python's Pandas) to identify behavioral patterns and emotional fluctuations.

[1696] Server: The analysis results are visually displayed using a visualization tool (e.g., Tableau), allowing companies to easily understand them and utilize them in their business strategies.

[1697] As a specific example, the analysis results can be displayed on a dashboard as graphs and charts, and can be viewed by companies.

[1698] Here's an example prompt: "Analyze the following social media data (posts, likes, and comments) and generate a user profile. Then, create a virtual avatar of the user based on this profile, track the avatar's behavior, and visualize the data."

[1699] This system allows the creation of detailed user profiles, real-time tracking and analysis of virtual avatar behavior, and the provision of a system that allows companies to effectively utilize the data. Furthermore, the introduction of an emotion engine enables advanced data analysis that reflects the user's emotional state.

[1700] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1701] Step 1: Data collection

[1702] User: The user launches a dedicated application and clicks a button to connect to the communication network service. They enter their login information on the displayed authentication screen and grant the application permission to access data. The input is the user's login information, and the output is an authentication token.

[1703] Specifically, the user clicks the SNS link button and enters their login information on the SNS's OAuth authentication page. This operation obtains an authentication token from the SNS.

[1704] Server: Using the authentication token, it sends a request to the API of the communication network service to obtain user data. The input is the authentication token and the API request, and the output is user data (posts, comments, images, likes, etc.).

[1705] Specifically, the server calls the SNS API to retrieve the user's latest posts and likes. This data is used in the subsequent analysis steps.

[1706] Step 2: Data Preprocessing

[1707] Server: Preprocesses the acquired user data. The input is raw data and the output is preprocessed data. Data cleaning involves removing unnecessary tags and special characters from text data and normalizing image data.

[1708] Specifically, the system strips out HTML tags from the text data, removes emojis and special characters, and converts the image data into a format suitable for analysis.

[1709] Step 3: Data analysis

[1710] Server: Analyzes the preprocessed data. The input is the preprocessed data, and the output is the analysis results (keywords, sentiment, tone, etc.). It uses natural language processing algorithms (e.g., BERT model) and sentiment engines (e.g., VADER) to extract information from text data. It also performs image analysis (e.g., YOLOv3).

[1711] Specifically, the BERT model is applied to text to extract keywords and perform sentiment analysis, and YOLOv3 is used to perform object recognition on image data.

[1712] Step 4: Generate a profile

[1713] Server: Generates a user profile based on the analysis results. The input is the analysis results, and the output is the user profile. The profile includes the user's interests, concerns, and emotional information.

[1714] Specifically, the extracted keywords and emotional information are compiled into profile data format and stored in a database.

[1715] Step 5: Creating a user avatar

[1716] Server: The generated profile information is input into a generative AI model (e.g., GPT-3) to generate the user's avatar in the virtual environment. The input is the user profile, and the output is the user's avatar (virtual character). The avatar reflects the user's characteristics and emotional state.

[1717] Specifically, profile data is input into GPT-3, and based on that, the appearance, personality, and behavioral characteristics of a virtual character are generated.

[1718] Step 6: Place your clones

[1719] Server: The generated avatar is placed at the initial position in the virtual environment. The input is the generated user avatar, and the output is the avatar in the virtual environment. The avatar is set to be able to act freely.

[1720] Specifically, the generated avatar is placed at a specific position in the virtual space using a virtual environment engine (e.g., Unity). This avatar is designed to interact with other avatars and elements.

[1721] Step 7: Collecting lifestyle data

[1722] Server: Activates the tracking system and tracks the behavior of the avatar in the virtual environment in real time. The input is the activity data of the avatar in the virtual environment, and the output is the behavior log. The behavior of the avatar also includes emotional information.

[1723] Specifically, the system collects logs of the avatar's movements, communications, and activities in real time and records them in a database.

[1724] Step 8: Data analysis and visualization

[1725] Server: Analyzes collected behavioral data to identify detailed user behavioral patterns, preferences, and emotional states. The input is behavioral data, and the output is the analysis results. Data analysis tools (e.g., Python's Pandas) are used to analyze the collected data.

[1726] Specifically, the collected data is analyzed using data analysis tools to identify behavioral patterns and emotional fluctuations.

[1727] Server: Visually displays the analysis results using a visualization tool (e.g., Tableau). The input is the analysis results, and the output is visualized data (graphs, charts, etc.).

[1728] Specifically, the analysis results are displayed on a dashboard as graphs and charts, allowing companies to easily understand the data and use it in their business strategies.

[1729] In this way, the present invention provides a system that generates detailed profiles of users, tracks and analyzes the behavior of their virtual avatars in real time, and allows companies to effectively utilize the data.

[1730] (Application example 2)

[1731] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1732] Conventional user behavior analysis systems using virtual spaces or social networking services (SNS) have had difficulty providing personalized content that effectively reflects users' interests and emotions. Furthermore, there has been a lack of systems that can collect users' emotional states in real time and use that data to recommend future content. This has limited the improvement of user experience.

[1733] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1734] In this invention, the server includes: means for a user to link with a social networking service; means for acquiring user data from the social networking service; means for analyzing the user data and creating a user profile; means for generating an avatar of the user in a virtual space based on the profile; means for collecting behavioral data of the user's avatar in the virtual space; means for analyzing the collected behavioral data; means for visualizing the analysis results; means for inserting the generated avatar of the user into video content; means for recommending video content based on the user profile; means for collecting emotional data of the user while watching a video; and means for analyzing the collected emotional data and reflecting it in the next video content recommendation. This makes it possible to provide personalized content that effectively reflects the user's interests and emotions and to improve the user experience.

[1735] A "social networking service" is an online platform that allows users to communicate with each other and share information via the Internet.

[1736] "User Data" refers collectively to user posts, comments, likes, images, and their history information collected on social networking services.

[1737] A "profile" is a set of information generated based on user data, including characteristics such as a user's interests, concerns, behavioral patterns, and emotional state.

[1738] A "virtual space" is a digital world that is generated using computer graphics and other techniques, and in which users can participate interactively.

[1739] A "user's avatar" is a digital avatar that imitates the user and is generated in a virtual space based on the user's profile.

[1740] "Behavioral data" is a record of the actions, movements, and emotional state of the user's avatar in the virtual space.

[1741] "Analysis results" is a general term for insights and information obtained after analyzing collected user data and behavioral data.

[1742] "Visualization" is a technique for visually expressing analytical results in a form that is easy to understand.

[1743] "Video content" is a general term for video and audio distributed over the Internet.

[1744] "Emotional data" is information about a user's emotional state that is analyzed from the user's posts and behavior.

[1745] The following describes an embodiment of the present invention. This system collects and analyzes a user's social networking service (SNS) data to create a user profile and provides personalized video content based on the results. Specifically, it inserts an avatar created using the user data into the video content, and collects emotional data from the user while watching the video to use in recommending the next content.

[1746] The server uses the following hardware and software:

[1747] Hardware:

[1748] Smartphones (including iPhones and Android devices)

[1749] Servers (high-performance computers for analysis and data processing)

[1750] software:

[1751] Python (programming language)

[1752] TensorFlow (a library for implementing deep learning models)

[1753] NLTK (Natural Language Processing Toolkit)

[1754] Django (web framework)

[1755] RESTful API (interface for exchanging data)

[1756] After the user installs the application on their smartphone, the system operates as follows.

[1757] 1. Data Collection:

[1758] A user launches the application and links their social media account. This linking sends data such as the user's posts, comments, and like history to the server using the social media API. For example, a user opens the application, clicks the button to link their Instagram account, and enters their login information on the OAuth authentication screen. This operation retrieves the user's photo posts and comments via the Instagram API.

[1759] 2. Data Analysis:

[1760] The server preprocesses the collected social media data, cleaning the text data and normalizing the image data, then uses natural language processing algorithms and an emotion engine to extract keywords, emotions, and tones from the text data and identify user interests through image analysis.

[1761] 3. User avatar generation:

[1762] Using a generative AI model, the system generates a virtual avatar of the user based on the analyzed profile data, and the avatar can behave in a way that reflects the user's emotions and interests.

[1763] 4. Personalize your video content:

[1764] The server recommends optimal video content based on the user profile. Furthermore, the server inserts the generated avatar into the video and acts as a guide based on the user's interests. For example, if the user likes "travel," the server recommends videos related to travel, in which the user's avatar guides them around tourist spots.

[1765] 5. Collecting Emotional Data:

[1766] While the user is watching the video, the server collects emotional data in real time, which is recorded as post-viewing feedback and used to recommend content for the next time.

[1767] Below are some examples of prompts to use when analyzing collected data:

[1768] Example prompt sentence:

[1769] Extract the following information for a user's Instagram posts:

[1770] keyword

[1771] Emotional state

[1772] Frequent phrases for each keyword

[1773] Based on these prompts, useful information is extracted from the user's posts and used to personalize the video content. The app also takes into account the user's emotional state to recommend the most suitable videos, and the user's avatar acts as a guide within the video, providing a more immersive experience.

[1774] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1775] Program processing flow and detailed explanation of each step

[1776] Step 1:

[1777] Data collection

[1778] Input: The user launches the application and links their social media account.

[1779] Operation: After the user clicks the SNS link button, the device displays the OAuth authentication screen. The user enters their login information and data access permissions.

[1780] Output: SNS authentication token and permissions to access user data are obtained.

[1781] How it works: The server uses the obtained authentication token to send a request to the SNS API to retrieve data such as the user's posts, comments, and likes.

[1782] Step 2:

[1783] Data Preprocessing

[1784] Input: Retrieved social media data.

[1785] How it works: The server cleans the collected data by removing unnecessary tags and special characters, and also normalizes the image data.

[1786] Output: Cleaned text data and normalized image data.

[1787] What it does: Pass this clean data on to the next analysis step.

[1788] Step 3:

[1789] Data analysis

[1790] Input: Cleaned text data and normalized image data.

[1791] How it works: The server uses natural language processing algorithms (e.g., NLTK) and emotion engines to extract keywords, emotional states, and tone from text data. It also performs image analysis to extract features that identify user interests.

[1792] Output: User's keyword list, emotional state and areas of interest.

[1793] How it works: Generates a user profile based on the analysis results.

[1794] Step 4:

[1795] User avatar generation

[1796] Input: The generated user profile.

[1797] How it works: The server uses a generative AI model to generate a virtual avatar for the user based on the user's profile, which reflects the avatar's characteristics and emotional state.

[1798] Output: A virtual avatar of the user.

[1799] Movement: Set the avatar to be able to move freely within the virtual space.

[1800] Step 5:

[1801] Personalized video content

[1802] Input: User profile and virtual self.

[1803] How it works: The server recommends the most suitable video content based on the user profile and inserts the user's avatar into the video. The avatar appears as a navigator in the video, guiding the user through the content.

[1804] Output: Personalized video content.

[1805] What happens: Video is streamed to your device.

[1806] Step 6:

[1807] Collecting Emotional Data

[1808] Input: The video content that the user watches.

[1809] How it works: The device collects the user's emotions in real time while watching, providing post-viewing feedback.

[1810] Output: Real-time collected emotion data.

[1811] Operation: The server analyzes this emotional data and reflects it in the next video content recommendation.

[1812] Step 7:

[1813] Using Feedback

[1814] Input: Collected real-time emotion data.

[1815] How it works: The server analyzes the collected emotional data and reflects it in the next video content recommendation. Specifically, it adjusts the recommendation algorithm based on changes in the user's emotional state and reactions to specific content.

[1816] Output: Upcoming video content recommendation list.

[1817] What it does: Provides users with a list of upcoming video content recommendations.

[1818] Through these steps, the system provides personalized content that effectively reflects the user's interests and emotions.

[1819] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1820] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1821] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1822] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1823] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1824] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1825] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1826] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1827] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1828] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1829] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1830] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1831] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1832] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1833] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1834] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1835] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1836] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1837] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1838] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1839] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1840] The following is further disclosed regarding the above embodiment.

[1841] (Claim 1)

[1842] a means for a user to interact with a social networking service;

[1843] means for obtaining user data from a social networking service;

[1844] means for analyzing user data and creating a profile of the user;

[1845] A means for generating an avatar of the user in a virtual space based on the profile;

[1846] A means for collecting behavioral data of a user's avatar in a virtual space;

[1847] a means for analyzing the collected behavioral data;

[1848] A means of visualizing the analysis results;

[1849] A system including:

[1850] (Claim 2)

[1851] 10. The system of claim 1, further comprising means for analyzing the user data using a natural language processing algorithm.

[1852] (Claim 3)

[1853] 10. The system of claim 1, further comprising means for the user's avatar to interact with other avatars in the virtual space.

[1854] "Example 1"

[1855] (Claim 1)

[1856] A means for users to interact with the information sharing system;

[1857] A means for acquiring user data from an information sharing system;

[1858] means for analyzing user data and creating a profile of the user;

[1859] means for generating a representative for the user within the virtual environment based on the characteristics;

[1860] means for collecting activity data of a user's representative within the virtual environment;

[1861] a means for analyzing the collected activity data;

[1862] A means of visualizing the analysis results;

[1863] A system including:

[1864] (Claim 2)

[1865] 10. The system of claim 1, further comprising means for analyzing the user data using a natural language processing algorithm.

[1866] (Claim 3)

[1867] 10. The system of claim 1, further comprising means for the user's representative to interact with other representatives within the virtual environment.

[1868] "Application Example 1"

[1869] (Claim 1)

[1870] a means for a user to interact with a social networking service;

[1871] means for obtaining user data from a social networking service;

[1872] means for analyzing user data and creating a profile of the user;

[1873] A means for generating an avatar of the user in a virtual space based on the profile;

[1874] A means for collecting behavioral data of a user's avatar in a virtual space;

[1875] a means for analyzing the collected behavioral data;

[1876] A means of visualizing the analysis results;

[1877] A means for suggesting products to a user in a virtual store based on the user's profile;

[1878] A system including:

[1879] (Claim 2)

[1880] 10. The system of claim 1, further comprising means for analyzing the user data using a natural language processing algorithm.

[1881] (Claim 3)

[1882] 10. The system of claim 1, further comprising means for allowing the user's avatar to interact with other avatars in the virtual space and for personalizing product suggestions based thereon.

[1883] "Example 2: Combining Emotion Engines"

[1884] (Claim 1)

[1885] a means for a user to interface with a communications network service;

[1886] means for obtaining user data from a communication network service;

[1887] means for pre-processing the acquired user data;

[1888] means for analyzing user data and creating a profile of the user;

[1889] means for generating an avatar of the user within the virtual environment based on the profile;

[1890] means for collecting behavioral data of a user's avatar within the virtual environment;

[1891] a means for analyzing the collected behavioral data;

[1892] A means of visualizing the analysis results;

[1893] A system including:

[1894] (Claim 2)

[1895] 10. The system of claim 1, further comprising means for analyzing the user data using a language analysis algorithm.

[1896] (Claim 3)

[1897] 10. The system of claim 1, further comprising means for the user's avatar to interact with other avatars within the virtual environment.

[1898] "Application example 2 when combining emotion engines"

[1899] (Claim 1)

[1900] a means for a user to interact with a social networking service;

[1901] means for obtaining user data from a social networking service;

[1902] means for analyzing user data and creating a profile of the user;

[1903] A means for generating an avatar of the user in a virtual space based on the profile;

[1904] A means for collecting behavioral data of a user's avatar in a virtual space;

[1905] a means for analyzing the collected behavioral data;

[1906] A means of visualizing the analysis results;

[1907] A means for inserting the generated user avatar into video content;

[1908] A means for recommending video content based on a user's profile;

[1909] A means of collecting emotional data from users while they are watching videos;

[1910] A method for analyzing the collected emotional data and reflecting it in the next video content recommendation.

[1911] A system including:

[1912] (Claim 2)

[1913] 10. The system of claim 1, further comprising means for analyzing the user data using a natural language processing algorithm.

[1914] (Claim 3)

[1915] 10. The system of claim 1, further comprising means for the user's avatar to interact with other avatars in the virtual space. [Explanation of symbols]

[1916] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to interact with a social networking service; means for obtaining user data from a social networking service; means for analyzing user data and creating a profile of the user; A means for generating an avatar of the user in a virtual space based on the profile; A means for collecting behavioral data of a user's avatar in a virtual space; a means for analyzing the collected behavioral data; A means of visualizing the analysis results; A system including:

2. 10. The system of claim 1, further comprising means for analyzing the user data using a natural language processing algorithm.

3. The system of claim 1 further comprising means for the user's avatar to interact with other avatars in the virtual space.

Images