system

Abstract

Provide a system. [Solution] the means by which data relating to individuals is collected; A means for analyzing the collected data and extracting personal characteristics; A means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality; and a means for adapting the generated digital persona to multiple interfaces; A means of capturing user input and analyzing and responding to it in real time; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Until now, there has been no technology that can preserve and continue an individual's spirit or personality in the digital space after death. This has not adequately addressed the needs of bereaved families who have lost a loved one and want to reconnect with their loved one's spirit or preserve their own personality before death. This has resulted in the loss of spiritual connection, making it difficult for them to share memories of the deceased. There has also been a need for a means to preserve one's personality in advance in case of an accident or illness. To address this, a system is needed that can efficiently collect and analyze personal data, generate a digital personality, and adapt it to an interface. [Means for solving the problem]

[0005] The present invention provides a means for collecting personal data, analyzing the data, and extracting personal characteristics. It also includes a means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality. Furthermore, the system provides a means for adapting the generated digital personality to multiple interfaces, capturing user input, and analyzing and responding in real time, thereby enabling users to interact with the digital personalities of the deceased and their own. This allows bereaved family members to maintain a spiritual connection with the deceased and to save their own personalities in advance.

[0006] "Personal data" refers to information about an individual, such as photographs, videos, audio messages, and text data.

[0007] "Data Collection Instrument" refers to a device or software that allows a user to upload personal data using a specific terminal and convert it into a standard format.

[0008] "Data analysis means" refers to a device or software for analyzing collected data and extracting characteristics such as an individual's tone of voice, language, facial expressions, and behavioral patterns.

[0009] A "generative artificial intelligence model" refers to an algorithm or system that learns from extracted characteristics and generates an individual's digital personality.

[0010] "Digital personality" refers to a digital personality generated by a generative artificial intelligence model that reproduces an individual's characteristics.

[0011] "Interface adaptation means" refers to a device or software that allows the generated digital personality to adapt to multiple devices (smartphones, PCs, AR / VR devices, robots, etc.).

[0012] "User input capture means" refers to a device or software for capturing input (voice, text, motion, etc.) from a user in real time and transmitting it to a server.

[0013] "Real-time analysis means" refers to a device or software for analyzing input from a user and generating an appropriate response.

[0014] The term "response generation means" refers to a device or software for generating an appropriate response to an input from a user and presenting it to the user.

[0015] "Secure communication protocol" refers to a communication procedure (e.g., HTTPS) for securely sending and receiving data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system for collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement the present invention.

[0038] User Data Collection Phase

[0039] Data upload

[0040] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. This process is carried out through a dedicated application or web portal.

[0041] Data Conversion

[0042] The device processes the uploaded data locally and converts it into a standard format (JSON, XML, etc.) that is then formatted in a way that is suitable for subsequent analysis steps.

[0043] Data transmission

[0044] The device then sends the converted data to a server using a secure communication protocol such as HTTPS to ensure privacy and data security.

[0045] Data analysis and learning phase

[0046] Data reception

[0047] The server receives the data sent from the device, temporarily stores it in storage, and uses it for the analysis process.

[0048] Data analysis

[0049] The server analyzes the received data and extracts characteristics such as the user's tone of voice, language, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) and image recognition technologies.

[0050] Model learning

[0051] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, which then generates a digital personality that reflects the user's characteristics.

[0052] Save Model

[0053] The server stores the trained digital personality model in storage and periodically updates it, ensuring that the digital personality reflects the latest data.

[0054] Digital personality generation phase

[0055] Digital personality generation

[0056] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[0057] Interface Preparation

[0058] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[0059] Interaction Phase

[0060] Receiving User Input

[0061] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[0062] Input Transmission

[0063] The terminal transmits the captured user input data to the server in real time.

[0064] Response Generation

[0065] The server analyzes the received input and generates an appropriate response, using a pre-trained digital personality model in the response generation process.

[0066] Response Send

[0067] The server generates a response and sends it to the device in an appropriate format (e.g., an audio file or text data).

[0068] Response suggestion

[0069] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[0070] Specific examples

[0071] Specific examples of remote conversations

[0072] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[0073] 2. The device converts the speech into text and sends it to the server.

[0074] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[0075] 4. The device converts the generated text into speech and conveys it to the user.

[0076] Specific examples of conversations using AR devices

[0077] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[0078] 2. The device captures the user's voice and movements and sends them to the server.

[0079] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[0080] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[0081] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[0082] The processing flow will be explained below.

[0083] User Data Collection Phase

[0084] Step 1:

[0085] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[0086] Step 2:

[0087] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[0088] Step 3:

[0089] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[0090] Data analysis and learning phase

[0091] Step 4:

[0092] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[0093] Step 5:

[0094] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[0095] Step 6:

[0096] The server uses the extracted characteristics information to train a user's digital personality using a generative artificial intelligence model. The training process uses a neural network and involves repeated training.

[0097] Step 7:

[0098] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[0099] Digital personality generation phase

[0100] Step 8:

[0101] The server uses the user's data and a trained AI model to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics.

[0102] Step 9:

[0103] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[0104] Interaction Phase

[0105] Step 10:

[0106] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[0107] Step 11:

[0108] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[0109] Step 12:

[0110] The server analyzes the received input data and generates an optimal response based on the user's words and actions, using a pre-trained digital personality model.

[0111] Step 13:

[0112] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[0113] Step 14:

[0114] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[0115] This specific processing flow allows the system to receive input from the user in real time and realize natural interactions with the digital personality that responds appropriately.

[0116] Example 1

[0117] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] While modern technology exists to collect and analyze personal data to generate digital personalities, the efficiency and security of the data conversion, transmission, and analysis processes may not be sufficiently ensured. Furthermore, the generated digital personalities have limited flexibility in adapting to multiple interfaces and limited ability to respond to user input in real time. This not only results in a poor user experience, but also poses privacy risks.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0120] In this invention, the server includes: means for collecting personal data; means for converting the collected data into a standard format (JSON, XML); means for transmitting the converted data using a secure communication protocol (HTTPS); means for receiving the transmitted data and temporarily storing it in storage; means for analyzing the stored data and extracting personal characteristics using natural language processing or image recognition; means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality; means for storing the generated digital personality in storage and periodically updating it; means for adapting the generated digital personality to multiple interfaces; means for capturing input from a user, analyzing it in real time, and generating a response; means for transmitting the generated response to a terminal and presenting it to the user; and means for transmitting the generated response to the terminal and presenting it to the user. This enables efficient and secure data conversion and transmission, enables the generated digital personality to flexibly adapt to multiple interfaces, and enables immediate and appropriate responses to real-time user input.

[0121] "Personal Data" refers to information about a user, such as a user's photos, videos, voice messages, text data, etc.

[0122] "Standard formats" refer to general-purpose data formats such as JSON and XML that are used to make data compatible between different systems.

[0123] A "secure communication protocol" is a protocol that ensures the confidentiality and integrity of data when sending and receiving information, and examples include HTTPS.

[0124] "Storage" refers to physical or virtual storage devices for storing data, including cloud storage and local disk drives.

[0125] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language; examples include speech recognition and text analysis.

[0126] "Image recognition technology" is a technology that recognizes and analyzes patterns and features in images, and examples include facial recognition and object detection.

[0127] A "generative artificial intelligence model" is an artificial intelligence model that generates new data and responses based on user data, and an example is GPT-3 (registered trademark).

[0128] A "digital persona" is a computer-generated virtual personality that reflects the user's characteristics and can interact with the user through voice, text, video, etc.

[0129] An "interface" is a means or device that allows a user to interact with a system, and includes smartphones, PCs, AR / VR devices, robots, etc.

[0130] "User input" refers to data such as voice, text, and actions that a user provides to the system.

[0131] "Real-time analysis" refers to the process of instantly analyzing input from a user and generating a response without delay.

[0132] A "response" is a reply or reaction that a system generates in response to input from a user, and may be in the form of audio, text, or video.

[0133] The present invention is a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. This system enables natural dialogue based on the user's data and maintains a psychological connection. The following describes how the present invention is specifically implemented.

[0134] User Data Collection Phase

[0135] Data upload

[0136] Users upload their own photos, videos, voice messages, and text data using dedicated applications or web portals. Specific hardware used includes smartphones (e.g., iPhone (registered trademark), Android (registered trademark)) and PCs (e.g., Windows PCs, Macs). Software used includes dedicated data upload applications (e.g., digital personality apps) and web portals.

[0137] Data Conversion

[0138] The terminal receives the uploaded data and converts it into a standard format (JSON, XML). This process uses Python and its libraries (e.g., Pandas, json library). For example, a JPEG photo uploaded by a user is BASE64 encoded and converted into JSON format.

[0139] Data transmission

[0140] The terminal transmits the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[0141] Data analysis and learning phase

[0142] Data reception

[0143] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[0144] Data analysis

[0145] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.). This analysis uses natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TENSORFLOW (registered trademark)).

[0146] Model learning

[0147] The server then trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user, using frameworks such as PyTorch and TensorFlow.

[0148] Save Model

[0149] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[0150] Digital personality generation phase

[0151] Digital personality generation

[0152] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[0153] Interface Preparation

[0154] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[0155] Interaction Phase

[0156] Receiving User Input

[0157] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[0158] Input Transmission

[0159] The device sends the captured user input data to the server using HTTPS.

[0160] Response Generation

[0161] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[0162] Response Send

[0163] The server sends the generated response to the terminal in an appropriate format, for example, as an audio file or text data.

[0164] Response suggestion

[0165] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[0166] Specific examples

[0167] Specific examples of remote conversations

[0168] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[0169] 2. The device converts the speech into text and sends it to the server.

[0170] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[0171] 4. The device converts the generated text into speech and conveys it to the user.

[0172] Specific examples of conversations using AR devices

[0173] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[0174] 2. The device captures the user's voice and movements and sends them to the server.

[0175] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[0176] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[0177] Example prompt sentence:

[0178] "Dad, how was the meeting today?"

[0179] "Mom, what did you have for lunch today?"

[0180] "How was your day? Anything new?"

[0181] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[0182] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0183] Step 1: Upload data

[0184] Users can upload their photos, videos, voice messages, and text data using a dedicated application or web portal. Specifically, they open the application, click the "Upload Data" button, select a file, and then press the "Send" button.

[0185] Input: Personal data on your smartphone or PC (photos, videos, voice messages, text data)

[0186] Output: Uploaded personal data is sent to the device.

[0187] Step 2: Data conversion

[0188] The terminal receives the uploaded data and converts it into a standard format (JSON, XML) using Python and its libraries (e.g., Pandas, json library).

[0189] Specific operation: For example, the terminal encodes a JPEG image using BASE64 and converts it into JSON format.

[0190] Input: Uploaded raw data (JPEG, MP4, WAV, TXT, etc.)

[0191] Output: Data converted to a standard format (BASE64 encoded JSON file)

[0192] Step 3: Send data

[0193] The terminal sends the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[0194] Specific operation: Include the converted data in the payload portion of the HTTPS request and post it to the server's API endpoint.

[0195] Input: Data converted to a standard format (BASE64 encoded JSON file)

[0196] Output: Data sent to the server via secure communication

[0197] Step 4: Receiving Data

[0198] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[0199] Specific operation: The server receives the HTTPS request, analyzes the payload portion, and executes instructions to save it in storage.

[0200] Input: Data received via HTTPS

[0201] Output: Data saved in storage

[0202] Step 5: Data analysis

[0203] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.) using natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TensorFlow).

[0204] Specific operation: The server converts, for example, an audio file into text, and analyzes the text to extract emotions and keywords.

[0205] Input: Data stored in storage

[0206] Output: User characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.)

[0207] Step 6: Model training

[0208] The server trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user. Frameworks such as PyTorch and TensorFlow are used.

[0209] Specific operation: The server inputs the user's characteristic data into the model and optimizes the model parameters.

[0210] Input: User characteristics data

[0211] Output: A trained generative artificial intelligence model

[0212] Step 7: Save the model

[0213] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[0214] Specific operation: The server periodically loads the trained model, retrains it with new data, and overwrites the updated model in the same storage.

[0215] Input: A trained generative artificial intelligence model

[0216] Output: The trained model saved in storage.

[0217] Step 8: Digital Persona Generation

[0218] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[0219] Specific operation: The server uses GPT-3 to generate phrases that the user might say, and then converts that text into speech using WaveNet.

[0220] Input: trained generative artificial intelligence model and user characteristics

[0221] Output: Response data such as audio, text, or video

[0222] Step 9: Prepare the interface

[0223] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[0224] Specific operation: The server uses the API of each device to convert voice data and text data into a format that can be distributed.

[0225] Input: Voice and text data

[0226] Output: Data adapted to each device

[0227] Step 10: Receiving User Input

[0228] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[0229] Specific operation: The device uses a microphone to capture the user's voice and processes the voice data in real time.

[0230] Input: Voice, text, and motion data from the user

[0231] Output: Captured input data

[0232] Step 11: Send Input

[0233] The device sends the captured user input data to the server using HTTPS.

[0234] Specific operation: The device encodes the captured audio data into BASE64 and sends it via an HTTPS request.

[0235] Input: Captured input data

[0236] Output: Input data sent to the server via secure communication

[0237] Step 12: Response Generation

[0238] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[0239] Specific operation: The server analyzes the user's input text and generates a response text based on its content using GPT-3.

[0240] Input: Captured user input data

[0241] Output: Generated response data

[0242] Step 13: Send response

[0243] The server sends the generated response to the terminal in an appropriate format, such as an audio file or text data.

[0244] Specific operation: The server converts the generated response text into audio using WaveNet and sends the audio file via HTTPS.

[0245] Input: Generated response data

[0246] Output: Response data sent to the terminal

[0247] Step 14: Present the response

[0248] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[0249] Specific operation: The terminal plays the requested audio data and lets the user listen to it.

[0250] Input: Response data received from the server

[0251] Output: The response presented to the user

[0252] (Application example 1)

[0253] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0254] Current digital security systems lack effective means to prevent unauthorized access. In particular, they do not adequately detect anomalies based on user behavior patterns, which increases the risk of unauthorized access and fraud. In addition, technology for managing user security using digital personas is not advanced, which prevents them from providing effective warnings and prevention measures.

[0255] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0256] In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from users and analyzing and responding in real time, and means for monitoring user behavior patterns to detect unauthorized access and issuing an alert when an abnormality is detected. This enables advanced anomaly detection based on user behavior patterns, making it possible to effectively prevent unauthorized access and fraud.

[0257] "Personal data" is information that can be associated with a specific individual, such as name, age, gender, voice, photograph, and behavioral patterns.

[0258] "Analyzing data and extracting individual characteristics" means using collected data and natural language processing and image recognition technology to reveal characteristics such as an individual's tone of voice and behavioral patterns.

[0259] A "generative artificial intelligence model" is an AI algorithm designed to perform specific tasks or responses based on collected data and analytical results.

[0260] A "digital personality" is a virtual personality formed by a generative artificial intelligence model that mimics the characteristics and behavioral patterns of a target individual.

[0261] "Multiple interfaces" refers to different platforms and devices, such as smartphones, PCs, AR / VR devices, and robots.

[0262] "User input" refers to various data that users provide to the system, such as voice, text, actions, and access logs.

[0263] "Analyzing and responding in real time" means instantly processing input from the user and generating and presenting an appropriate response.

[0264] "Monitoring user behavior patterns to detect unauthorized access" means analyzing user access logs and behavioral data to identify suspicious behavior that differs from normal behavior.

[0265] "Issuing a warning when an abnormality is detected" means that if the system detects any behavior that it deems to be fraudulent, it will immediately notify the user of the warning.

[0266] This invention relates to a security system that detects unauthorized access by collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement this invention.

[0267] User Data Collection Phase

[0268] Data upload

[0269] Users use smartphones or PCs to upload their own photos, voice messages, text data, and movement information through a dedicated application, and through this process, a variety of personal data is collected.

[0270] Data Conversion

[0271] The device locally converts the uploaded data into a standard format (JSON, XML, etc.) that is easy to parse and can be used in subsequent steps.

[0272] Data transmission

[0273] The converted data from the device is sent to the server using a secure protocol such as HTTPS, which ensures the privacy and security of the data.

[0274] Data analysis and learning phase

[0275] Data reception

[0276] The server receives the data sent from the terminal and temporarily stores it in storage.

[0277] Data analysis

[0278] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data and extract characteristics such as the user's tone of voice, vocabulary, and behavioral patterns.

[0279] Model learning

[0280] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, resulting in a digital personality model that reflects the user's characteristics.

[0281] Save Model

[0282] The server stores the learned digital personality model in storage and periodically updates it as needed.

[0283] Digital personality generation phase

[0284] Digital personality generation

[0285] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[0286] Interface Preparation

[0287] The server configures the generated digital personality to adapt to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots.

[0288] Interaction Phase

[0289] Receiving User Input

[0290] The device captures input from the user (voice, text, actions, etc.), and access logs and behavioral data are collected to detect unauthorized access.

[0291] Input Transmission

[0292] The terminal transmits the captured user input data to the server in real time.

[0293] Response Generation

[0294] The server analyzes the input it receives and generates an appropriate response, using a pre-trained model of the digital personality in the response generation process, along with algorithms designed to detect unauthorized access.

[0295] Response Send

[0296] The server generates a response and sends it to the terminal in the appropriate format.

[0297] Response suggestion

[0298] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[0299] Specific examples

[0300] Security system examples

[0301] 1. A user tries to unlock their smartphone.

[0302] 2. The device captures the user's operations and voice and sends the data to the server.

[0303] 3. The server analyzes the user's operation patterns and voice data to check for signs of unauthorized access. For example, if the user says, "This number is not mine."

[0304] 4. If the server detects an abnormality, it will send an alert to the terminal and display a response to the user saying "Security warning: Unauthorized access has been detected. Please check immediately."

[0305] This enables advanced anomaly detection based on user behavior patterns, effectively preventing unauthorized access and fraud.

[0306] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0307] Step 1:

[0308] Users upload personal data such as photos, voice messages, text data, and motion information from their smartphones or PCs through a dedicated application. The input data is diverse and includes, for example, the user's latest profile picture and recent conversation history. This data is converted into a standard format (such as JSON or XML) on the local device.

[0309] Step 2:

[0310] The device then sends the converted data to the server via a secure communication protocol (HTTPS), including the user's photos, voice messages, text data, and motion information, ensuring data privacy and security.

[0311] Step 3:

[0312] The server receives the data sent from the device and temporarily stores it in storage. The received data format is standard JSON or XML.

[0313] Step 4:

[0314] The server analyzes the received data and extracts characteristics such as the user's tone of voice, vocabulary, and behavioral patterns. These characteristics are analyzed using natural language processing (NLP) and image recognition technology. For example, NLP technology can be used to analyze the user's speaking style from a voice message, and image recognition technology can be used to analyze the user's facial expressions from a photograph.

[0315] Step 5:

[0316] The server trains a generative artificial intelligence model based on the extracted characteristics to generate a digital personality. This digital personality model reflects the user's characteristics and has the ability to respond via voice and text. The generated digital personality is stored in storage and updated periodically as needed.

[0317] Step 6:

[0318] The server configures the generated digital persona to adapt to multiple interfaces (e.g., smartphones, PCs, AR / VR devices, robots), providing a consistent digital persona across all platforms.

[0319] Step 7:

[0320] When a user tries to unlock their smartphone, the device captures the user's actions and voice. The input data includes, for example, the timing of the unlock operation and the voice message. The captured data is sent to the server in real time.

[0321] Step 8:

[0322] The server analyzes the received user input data and checks for signs of unauthorized access. It uses an AI model to analyze behavioral patterns and generates a warning if an anomaly is detected. For example, it is designed to detect an anomaly if a user says, "This number is not mine."

[0323] Step 9:

[0324] If the server detects an abnormality, it will send a warning to the device. This warning will be in the form of "Security Alert: Unauthorized access has been detected. Please check immediately."

[0325] Step 10:

[0326] The device will present the warning received from the server to the user by outputting audio in the case of voice, displaying on the screen in the case of text, and outputting both audio and video in the case of AR / VR devices and robots.

[0327] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0328] The present invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more human-like responses. The following embodiments explain how the present invention is specifically implemented.

[0329] User Data Collection Phase

[0330] Data upload

[0331] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[0332] Data Conversion

[0333] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[0334] Data transmission

[0335] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[0336] Data analysis and learning phase

[0337] Data reception

[0338] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[0339] Data analysis

[0340] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[0341] emotion recognition

[0342] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[0343] Model learning

[0344] The server then uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[0345] Save Model

[0346] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[0347] Digital personality generation phase

[0348] Digital personality generation

[0349] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[0350] Interface Preparation

[0351] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[0352] Interaction Phase

[0353] Receiving User Input

[0354] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[0355] Input Transmission

[0356] The device transmits the captured user input data to the server in real time, and may convert the voice data into text data before transmitting.

[0357] emotion recognition

[0358] The server analyzes the user's emotional state from the received input data and generates a response based on the analysis results.

[0359] Response Generation

[0360] The server analyzes the received input data and emotional information and generates optimal responses based on the user's words and actions, using a pre-trained digital personality model and emotion engine.

[0361] Response Send

[0362] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[0363] Response suggestion

[0364] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[0365] Specific examples

[0366] Specific examples of remote conversations

[0367] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[0368] 2. The device converts the speech into text and sends it to the server.

[0369] 3. The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[0370] 4. The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[0371] 5. The device converts the generated text into speech and conveys it to the user.

[0372] Specific examples of conversations using AR devices

[0373] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[0374] 2. The device captures the user's voice and movements and sends them to the server.

[0375] 3. The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[0376] 4. The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[0377] 5. The terminal presents the generated response to the user via audio and video.

[0378] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[0379] The processing flow will be explained below.

[0380] User Data Collection Phase

[0381] Step 1:

[0382] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[0383] Step 2:

[0384] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[0385] Step 3:

[0386] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[0387] Data analysis and learning phase

[0388] Step 4:

[0389] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[0390] Step 5:

[0391] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[0392] emotion recognition

[0393] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[0394] Step 6:

[0395] The server uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[0396] Step 7:

[0397] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[0398] Digital personality generation phase

[0399] Step 8:

[0400] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[0401] Step 9:

[0402] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[0403] Interaction Phase

[0404] Step 10:

[0405] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[0406] Step 11:

[0407] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[0408] Step 12:

[0409] The server analyzes the received input data and uses an emotion engine to recognize the user's emotional state. It combines emotional information with text data to understand the user's intentions.

[0410] Step 13:

[0411] The server generates appropriate responses based on the user's emotional information, using a pre-trained digital personality model to create natural responses that match the user's emotions.

[0412] Step 14:

[0413] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[0414] Step 15:

[0415] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[0416] Specific examples

[0417] Specific examples of remote conversations

[0418] Step 1:

[0419] The user uses their smartphone to say, "Dad, I had a hard time at work today."

[0420] Step 2:

[0421] The device converts the speech into text and sends it to the server.

[0422] Step 3:

[0423] The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[0424] Step 4:

[0425] The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[0426] Step 5:

[0427] The terminal converts the generated text into speech and conveys it to the user.

[0428] Specific examples of conversations using AR devices

[0429] Step 1:

[0430] The user puts on an AR headset and talks to a digital mom in their living room.

[0431] Step 2:

[0432] The device captures the user's voice and movements and sends them to a server.

[0433] Step 3:

[0434] The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[0435] Step 4:

[0436] The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[0437] Step 5:

[0438] The terminal presents the generated response to the user audio and visually.

[0439] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[0440] Example 2

[0441] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0442] In today's world, there is a growing demand for systems that provide responses that understand the user's characteristics and emotions. However, existing systems lack the ability to collect and analyze personal data, making it difficult to accurately recognize the user's emotions and generate responses. Furthermore, while there is a need for systems that can support a variety of interfaces, there is a lack of efficient means for achieving this. The purpose of this invention is to solve the above problems.

[0443] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for converting the collected data into a standard format, means for analyzing the converted data and extracting individual characteristics, means for recognizing an emotional state from the extracted characteristics, means for training a generative artificial intelligence model based on the characteristic information and emotional information and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, and means for capturing input from a user and analyzing and responding in real time. This makes it possible to provide responses that accurately reflect the user's characteristics and emotions through a variety of interfaces.

[0444] "Personal data" refers to information about a user, such as photos, videos, voice messages, and text data.

[0445] "Converting to a standard format" refers to converting collected data into a common format such as JSON or XML.

[0446] "Extracting personal characteristics" means analyzing characteristic information such as the user's tone of voice, choice of words, facial expressions, and behavioral patterns from the received data.

[0447] "Recognizing emotional state" means identifying the user's emotions (joy, sadness, anger, surprise, etc.) from the analyzed data.

[0448] A "generative artificial intelligence model" is an artificial intelligence model that learns using machine learning techniques such as neural networks and generates responses based on the user's characteristics and emotions.

[0449] A "digital persona" is a virtual personality that reproduces the user's characteristics and emotions, generated based on collected data and generative artificial intelligence models.

[0450] "Adapting to multiple interfaces" means adjusting the generated digital personality so that it can be used on different platforms, such as smartphones, PCs, AR / VR devices, and robots.

[0451] "Analyzing and responding in real time" means instantly analyzing input from the user and instantly generating and sending back an appropriate response based on that input.

[0452] MODE FOR CARRYING OUT THE INVENTION

[0453] This invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it aims to achieve more human-like responses.

[0454] User Data Collection Phase

[0455] Users upload their photos, videos, voice messages, text data, etc. to a specific application or web portal via their smartphone or PC. The device receives the uploaded data and converts it locally into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[0456] Data transmission and analysis

[0457] The device sends the converted data to the server. A secure communication protocol such as HTTPS is used for transmission, ensuring data safety. The server receives the data sent from the device and temporarily stores it in storage. At this stage, verification is performed to confirm the data's integrity and consistency.

[0458] The server then analyzes the received data to extract characteristic information such as the user's tone of voice, vocabulary, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) techniques and image recognition algorithms. The server then uses an emotion engine to recognize the user's emotional state from the information obtained through data analysis. For example, it can determine whether the user is happy, sad, angry, or surprised.

[0459] Model training and saving

[0460] The server uses a generative artificial intelligence model to learn the user's digital personality based on the extracted characteristic information and emotional information. Deep learning technology is used to train the neural network. The learned digital personality model is stored in storage and is periodically updated with new data, ensuring that the digital personality reflects the latest information.

[0461] Digital personality generation and interface adaptation

[0462] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The generated digital personality faithfully reproduces the user's characteristics and emotions. The server also configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[0463] Interaction Phase

[0464] Users can interact with their digital persona through the device. The device captures input from the user (voice, text, actions, etc.) and sends it to the server in real time. The server then uses the received input data to analyze the user's emotional state using an emotion engine, and generates a response based on the analysis results. The generated response is sent from the server to the device and presented to the user.

[0465] Specific examples

[0466] Specific examples of remote conversations

[0467] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[0468] 2. The device converts the voice into text using automatic speech recognition (ASR) technology and sends it to the server.

[0469] 3. The server analyzes the received text using NLP algorithms and uses an emotion engine to recognize the user's emotions, such as stress or fatigue.

[0470] 4. The server generates a response based on the user's emotional information and past data: "That must have been tough. What happened?"

[0471] 5. The device converts the generated text into speech using text-to-speech (TTS) technology and outputs it through the speaker.

[0472] Specific examples of conversations using AR devices

[0473] 1. A user puts on an AR headset and talks to a virtual digital personality of their mother in their living room.

[0474] 2. The device captures the user's voice and movements using a camera and sensors and sends them to the server.

[0475] 3. The server analyzes the user's comments and actions using NLP and an emotion engine and determines that the user is happy.

[0476] 4. The server generates a response saying, "Wow, that's a nice photo. Where did you take it?"

[0477] 5. The device uses audio and AR images to present the response to the user.

[0478] The system allows users to interact naturally with their digital persona, maintaining an emotional connection through emotionally aware responses. Examples of prompts include "I'm so tired today. Tell me something uplifting," "Let's talk about what happened with your friend yesterday," and "Mom, what do you think about the news these days?"

[0479] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0480] Step 1: Upload data

[0481] Users use their smartphones or PCs to upload personal data such as photos, videos, voice messages, and text data to specific applications or web portals. These data are input, and the device receives the data.

[0482] Step 2: Data conversion

[0483] The terminal converts the uploaded data it receives into a standard format (e.g., JSON, XML). This conversion process includes changing the format of the data and adding metadata. The input raw data is reformatted, and the output is data in a standard format.

[0484] Step 3: Send data

[0485] The device sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure the data is secure. The input is data in a standard format, and the output is sent to the server.

[0486] Step 4: Data reception and storage

[0487] The server receives data sent from the device and temporarily stores it in storage. It verifies the received data to ensure its integrity and consistency. It also verifies the input data and outputs it as stored data.

[0488] Step 5: Data analysis

[0489] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns. The analysis uses natural language processing (NLP) technology and image recognition algorithms. The input data is analyzed and the extracted characteristic information is output.

[0490] Step 6: Emotion Recognition

[0491] The server uses an emotion engine based on the characteristic information obtained from the data analysis to recognize the user's emotional state, for example, joy, sadness, anger, surprise, etc. The input is the analyzed characteristic information, and the output is the recognized emotion information.

[0492] Step 7: Model training

[0493] The server uses a generative artificial intelligence model to train a digital personality based on the extracted trait and emotion information. This process uses deep learning techniques to train a neural network. The input is the trait and emotion information, and the output is the trained model.

[0494] Step 8: Save the model

[0495] The server saves the trained digital personality model in storage, and periodically updates the model with new data. The saved model is obtained as output.

[0496] Step 9: Digital Persona Generation

[0497] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The input data is the user's real-time information, and the output is a response.

[0498] Step 10: Interface Adaptation

[0499] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converts data formats, and generates output appropriate for each interface.

[0500] Step 11: Receiving User Input

[0501] The device captures input from the user (voice, text, movement, etc.). When using an AR device or robot as an action, the input is obtained using a camera or sensor. The captured data becomes the input.

[0502] Step 12: Send Input

[0503] The device sends the captured user input data to the server in real time. In the case of voice data, it is converted to text using ASR technology before being sent. The input is the captured data, and the data sent to the server is the output.

[0504] Step 13: Response Generation

[0505] The server uses the digital personality to generate the optimal response based on the input data and emotional information it receives. Analysis and response generation are performed based on the input data, and the generated response is obtained as output.

[0506] Step 14: Send response

[0507] The server converts the generated response into an appropriate format (audio file, text data, etc.) and sends it to the device. An optimized response is generated as output and sent to the device.

[0508] Step 15: Present the response

[0509] The device presents the response received from the server to the user. Voice responses are output from the speaker, and text responses are displayed on the screen. AR / VR devices use audio and video for output. The presented response is the final output.

[0510] (Application example 2)

[0511] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0512] Conventional customer service systems have difficulty in responding to customer emotions and personal characteristics, making it difficult to achieve human-like dialogue. This has made it difficult to improve customer satisfaction in brick-and-mortar stores. It has also been difficult to provide appropriate product introductions and personalized services.

[0513] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from customers and analyzing and responding in real time, means for recognizing emotions and generating appropriate responses, and means for providing customer service. This enables more human-like interactions and service provision that reflects the customer's emotions and personal characteristics.

[0514] "Personal data" refers to information such as an individual's photos, videos, voice messages, and text data.

[0515] "Means of collection" refers to the methods and devices used to obtain data using devices such as smartphones and PCs.

[0516] "Means of analysis" refers to the technology used to analyze the acquired data and extract individual characteristics and behavioral patterns.

[0517] "Personal characteristics" refer to an individual's unique characteristics, such as tone of voice, language, facial expressions, and behavioral patterns.

[0518] A "generative artificial intelligence model" refers to an artificial intelligence that learns based on collected data and has specific functions and capabilities.

[0519] "Digital personas" refer to digital agents created using generative artificial intelligence models that replicate human characteristics and emotions.

[0520] "Means of adaptation" refers to technology that allows the generated digital personality to be compatible with multiple interfaces (smartphones, PCs, AR / VR devices, etc.).

[0521] "Means for capturing input and analyzing and responding in real time" refers to technology that receives user voice or text input, analyzes it immediately, and generates an appropriate response.

[0522] "Means for performing emotion recognition and generating appropriate responses" refers to technology that detects the user's emotional state and generates an optimal response based on that emotion.

[0523] "Means for providing customer service" refers to methods and technologies for providing product information and personalized services to customers in physical stores.

[0524] "Customer" refers to consumers or users who visit a physical store.

[0525] This invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more human-like responses.

[0526] System configuration and program processing

[0527] Hardware and Software:

[0528] Hardware: Smartphones, smart glasses, PCs, servers

[0529] Software: Python, TensorFlow, DeepFace, Transformers (Hugging Face)

[0530] The server includes means for collecting and analyzing personal data, means for generating a digital personality using a generative artificial intelligence model, means for adapting to multiple interfaces, means for capturing input from customers and analyzing and responding in real time, means for performing emotion recognition, and means for providing customer service.

[0531] Program processing procedure

[0532] Data collection:

[0533] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, and more to an application or web portal.

[0534] Data conversion and transmission:

[0535] The device converts the uploaded data into a standard format (e.g., JSON, XML) and sends it to the server using a secure communication protocol, encrypting the data before sending it.

[0536] Data analysis and model training:

[0537] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) technology and image recognition algorithms. Based on the extracted characteristics and emotional information, a generative AI model is trained to generate a digital personality.

[0538] Digital personality and emotion recognition:

[0539] The server combines the generated digital personality with an emotion engine to recognize the user's emotions in real time and generate corresponding responses, using TensorFlow and DeepFace for this process.

[0540] Interface Adaptation:

[0541] The generated digital personality is adapted to multiple interfaces such as smartphones, smart glasses, and PCs.

[0542] Specific examples

[0543] Example 1: Customer service via smartphone

[0544] 1. A user uses their smartphone to ask, "What are the features of this product?"

[0545] 2. The device converts the speech into text and sends it to the server.

[0546] 3. The server analyzes the text and uses an emotion engine to determine which content the user is interested in.

[0547] 4. The server generates the appropriate response: "This product has a modern design and is particularly durable."

[0548] 5. The device converts the generated response into speech and conveys it to the user.

[0549] Example 2: Customer service using smart glasses

[0550] 1. The user asks through the smart glasses, "How do I use this product?"

[0551] 2. The device captures audio and movement and sends it to the server.

[0552] 3. The server analyzes the user's question and uses an emotion engine to recognize their interest.

[0553] 4. The server generates the appropriate response: "Using this product is very easy. First..."

[0554] 5. The terminal presents the generated response to the user via audio and video.

[0555] Example prompt sentence:

[0556] Customer question: "What are the features of this product?"

[0557] Emotion: "Excited"

[0558] Generate responses for your digital persona.

[0559] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0560] Step 1: Data collection

[0561] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to an application or web portal. The input data becomes the raw material for analyzing and extracting personal characteristics.

[0562] Step 2: Data conversion

[0563] The device receives the uploaded data, converts it into a standard format (e.g., JSON, XML), adds metadata, and conforms to the format, and then transmits the converted data to the server using a secure communication protocol (e.g., HTTPS).

[0564] Step 3: Data reception and analysis

[0565] The server receives the converted data sent from the device and temporarily stores it in storage. Based on the input data, it uses natural language processing (NLP) technology and image recognition algorithms to analyze and extract characteristics such as the user's tone of voice, vocabulary, facial expressions, and behavioral patterns.

[0566] Step 4: Emotion Recognition

[0567] The server uses the information obtained from the data analysis to analyze the user's emotional state using an emotion engine, identifying the type of emotion (e.g., joy, sadness, anger, etc.), and further processing the data based on the analysis results.

[0568] Step 5: Model training and digital personality generation

[0569] The server trains a generative AI model based on the extracted characteristics and emotional information to generate a digital personality. The training process uses neural network technology (e.g., TensorFlow) to train the model. The generated digital personality model is then stored in storage.

[0570] Step 6: Interface Adaptation

[0571] The server adapts the generated digital personality to multiple interfaces (smartphones, smart glasses, PCs, etc.), optimizing its response and display formats for each device.

[0572] Step 7: Receiving User Input

[0573] When a user sends a question or request through a smartphone or smart glasses, the device captures the input (voice, text, movement, etc.) and converts the voice data to text as needed before transmitting the input to the server in real time.

[0574] Step 8: Response Generation

[0575] The server analyzes the user's input data and emotional information and generates an appropriate response. This process again uses generative AI models and an emotion engine. The appropriate response is converted into text or voice.

[0576] Step 9: Send and present the response

[0577] The server converts the generated response into an appropriate format (audio file, text data) and sends it to the device, which then presents the received response to the user. In the case of smartphones or smart glasses, the response is provided as audio or displayed on a screen.

[0578] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0579] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0580] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0581] [Second embodiment]

[0582] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0583] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0584] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0585] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0586] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0587] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0588] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0589] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0590] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0593] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0594] The present invention relates to a system for collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement the present invention.

[0595] User Data Collection Phase

[0596] Data upload

[0597] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. This process is carried out through a dedicated application or web portal.

[0598] Data Conversion

[0599] The device processes the uploaded data locally and converts it into a standard format (JSON, XML, etc.) that is then formatted in a way that is suitable for subsequent analysis steps.

[0600] Data transmission

[0601] The device then sends the converted data to a server using a secure communication protocol such as HTTPS to ensure privacy and data security.

[0602] Data analysis and learning phase

[0603] Data reception

[0604] The server receives the data sent from the device, temporarily stores it in storage, and uses it for the analysis process.

[0605] Data analysis

[0606] The server analyzes the received data and extracts characteristics such as the user's tone of voice, language, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) and image recognition technologies.

[0607] Model learning

[0608] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, which then generates a digital personality that reflects the user's characteristics.

[0609] Save Model

[0610] The server stores the trained digital personality model in storage and periodically updates it, ensuring that the digital personality reflects the latest data.

[0611] Digital personality generation phase

[0612] Digital personality generation

[0613] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[0614] Interface Preparation

[0615] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[0616] Interaction Phase

[0617] Receiving User Input

[0618] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[0619] Input Transmission

[0620] The terminal transmits the captured user input data to the server in real time.

[0621] Response Generation

[0622] The server analyzes the received input and generates an appropriate response, using a pre-trained digital personality model in the response generation process.

[0623] Response Send

[0624] The server generates a response and sends it to the device in an appropriate format (e.g., an audio file or text data).

[0625] Response suggestion

[0626] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[0627] Specific examples

[0628] Specific examples of remote conversations

[0629] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[0630] 2. The device converts the speech into text and sends it to the server.

[0631] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[0632] 4. The device converts the generated text into speech and conveys it to the user.

[0633] Specific examples of conversations using AR devices

[0634] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[0635] 2. The device captures the user's voice and movements and sends them to the server.

[0636] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[0637] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[0638] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[0639] The processing flow will be explained below.

[0640] User Data Collection Phase

[0641] Step 1:

[0642] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[0643] Step 2:

[0644] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[0645] Step 3:

[0646] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[0647] Data analysis and learning phase

[0648] Step 4:

[0649] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[0650] Step 5:

[0651] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[0652] Step 6:

[0653] The server uses the extracted characteristics information to train a user's digital personality using a generative artificial intelligence model. The training process uses a neural network and involves repeated training.

[0654] Step 7:

[0655] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[0656] Digital personality generation phase

[0657] Step 8:

[0658] The server uses the user's data and a trained AI model to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics.

[0659] Step 9:

[0660] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[0661] Interaction Phase

[0662] Step 10:

[0663] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[0664] Step 11:

[0665] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[0666] Step 12:

[0667] The server analyzes the received input data and generates an optimal response based on the user's words and actions, using a pre-trained digital personality model.

[0668] Step 13:

[0669] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[0670] Step 14:

[0671] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[0672] This specific processing flow allows the system to receive input from the user in real time and realize natural interactions with the digital personality that responds appropriately.

[0673] Example 1

[0674] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] While modern technology exists to collect and analyze personal data to generate digital personalities, the efficiency and security of the data conversion, transmission, and analysis processes may not be sufficiently ensured. Furthermore, the generated digital personalities have limited flexibility in adapting to multiple interfaces and limited ability to respond to user input in real time. This not only results in a poor user experience, but also poses privacy risks.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0677] In this invention, the server includes: means for collecting personal data; means for converting the collected data into a standard format (JSON, XML); means for transmitting the converted data using a secure communication protocol (HTTPS); means for receiving the transmitted data and temporarily storing it in storage; means for analyzing the stored data and extracting personal characteristics using natural language processing or image recognition; means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality; means for storing the generated digital personality in storage and periodically updating it; means for adapting the generated digital personality to multiple interfaces; means for capturing input from a user, analyzing it in real time, and generating a response; means for transmitting the generated response to a terminal and presenting it to the user; and means for transmitting the generated response to the terminal and presenting it to the user. This enables efficient and secure data conversion and transmission, enables the generated digital personality to flexibly adapt to multiple interfaces, and enables immediate and appropriate responses to real-time user input.

[0678] "Personal Data" refers to information about a user, such as a user's photos, videos, voice messages, text data, etc.

[0679] "Standard formats" refer to general-purpose data formats such as JSON and XML that are used to make data compatible between different systems.

[0680] A "secure communication protocol" is a protocol that ensures the confidentiality and integrity of data when sending and receiving information, and examples include HTTPS.

[0681] "Storage" refers to physical or virtual storage devices for storing data, including cloud storage and local disk drives.

[0682] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language; examples include speech recognition and text analysis.

[0683] "Image recognition technology" is a technology that recognizes and analyzes patterns and features in images, and examples include facial recognition and object detection.

[0684] A "generative artificial intelligence model" is an artificial intelligence model that generates new data and responses based on user data, and an example is GPT-3.

[0685] A "digital persona" is a computer-generated virtual personality that reflects the user's characteristics and can interact with the user through voice, text, video, etc.

[0686] An "interface" is a means or device that allows a user to interact with a system, and includes smartphones, PCs, AR / VR devices, robots, etc.

[0687] "User input" refers to data such as voice, text, and actions that a user provides to the system.

[0688] "Real-time analysis" refers to the process of instantly analyzing input from a user and generating a response without delay.

[0689] A "response" is a reply or reaction that a system generates in response to input from a user, and may be in the form of audio, text, or video.

[0690] The present invention is a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. This system enables natural dialogue based on the user's data and maintains a psychological connection. The following describes how the present invention is specifically implemented.

[0691] User Data Collection Phase

[0692] Data upload

[0693] Users upload their own photos, videos, voice messages, and text data using dedicated applications or web portals. Specific hardware used includes smartphones (e.g., iPhone, Android) and PCs (e.g., Windows PC, Mac). Software used includes dedicated data upload applications (e.g., digital personality apps) and web portals.

[0694] Data Conversion

[0695] The terminal receives the uploaded data and converts it into a standard format (JSON, XML). This process uses Python and its libraries (e.g., Pandas, json library). For example, a JPEG photo uploaded by a user is BASE64 encoded and converted into JSON format.

[0696] Data transmission

[0697] The terminal transmits the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[0698] Data analysis and learning phase

[0699] Data reception

[0700] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[0701] Data analysis

[0702] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.). This analysis uses natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TensorFlow).

[0703] Model learning

[0704] The server then trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user, using frameworks such as PyTorch and TensorFlow.

[0705] Save Model

[0706] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[0707] Digital personality generation phase

[0708] Digital personality generation

[0709] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[0710] Interface Preparation

[0711] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[0712] Interaction Phase

[0713] Receiving User Input

[0714] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[0715] Input Transmission

[0716] The device sends the captured user input data to the server using HTTPS.

[0717] Response Generation

[0718] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[0719] Response Send

[0720] The server sends the generated response to the terminal in an appropriate format, for example, as an audio file or text data.

[0721] Response suggestion

[0722] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[0723] Specific examples

[0724] Specific examples of remote conversations

[0725] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[0726] 2. The device converts the speech into text and sends it to the server.

[0727] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[0728] 4. The device converts the generated text into speech and conveys it to the user.

[0729] Specific examples of conversations using AR devices

[0730] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[0731] 2. The device captures the user's voice and movements and sends them to the server.

[0732] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[0733] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[0734] Example prompt sentence:

[0735] "Dad, how was the meeting today?"

[0736] "Mom, what did you have for lunch today?"

[0737] "How was your day? Anything new?"

[0738] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[0739] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0740] Step 1: Upload data

[0741] Users can upload their photos, videos, voice messages, and text data using a dedicated application or web portal. Specifically, they open the application, click the "Upload Data" button, select a file, and then press the "Send" button.

[0742] Input: Personal data on your smartphone or PC (photos, videos, voice messages, text data)

[0743] Output: Uploaded personal data is sent to the device.

[0744] Step 2: Data conversion

[0745] The terminal receives the uploaded data and converts it into a standard format (JSON, XML) using Python and its libraries (e.g., Pandas, json library).

[0746] Specific operation: For example, the terminal encodes a JPEG image using BASE64 and converts it into JSON format.

[0747] Input: Uploaded raw data (JPEG, MP4, WAV, TXT, etc.)

[0748] Output: Data converted to a standard format (BASE64 encoded JSON file)

[0749] Step 3: Send data

[0750] The terminal sends the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[0751] Specific operation: Include the converted data in the payload portion of the HTTPS request and post it to the server's API endpoint.

[0752] Input: Data converted to a standard format (BASE64 encoded JSON file)

[0753] Output: Data sent to the server via secure communication

[0754] Step 4: Receiving Data

[0755] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[0756] Specific operation: The server receives the HTTPS request, analyzes the payload portion, and executes instructions to save it in storage.

[0757] Input: Data received via HTTPS

[0758] Output: Data saved in storage

[0759] Step 5: Data analysis

[0760] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.) using natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TensorFlow).

[0761] Specific operation: The server converts, for example, an audio file into text, and analyzes the text to extract emotions and keywords.

[0762] Input: Data stored in storage

[0763] Output: User characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.)

[0764] Step 6: Model training

[0765] The server trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user. Frameworks such as PyTorch and TensorFlow are used.

[0766] Specific operation: The server inputs the user's characteristic data into the model and optimizes the model parameters.

[0767] Input: User characteristics data

[0768] Output: A trained generative artificial intelligence model

[0769] Step 7: Save the model

[0770] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[0771] Specific operation: The server periodically loads the trained model, retrains it with new data, and overwrites the updated model in the same storage.

[0772] Input: A trained generative artificial intelligence model

[0773] Output: The trained model saved in storage.

[0774] Step 8: Digital Persona Generation

[0775] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[0776] Specific operation: The server uses GPT-3 to generate phrases that the user might say, and then converts that text into speech using WaveNet.

[0777] Input: trained generative artificial intelligence model and user characteristics

[0778] Output: Response data such as audio, text, or video

[0779] Step 9: Prepare the interface

[0780] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[0781] Specific operation: The server uses the API of each device to convert voice data and text data into a format that can be distributed.

[0782] Input: Voice and text data

[0783] Output: Data adapted to each device

[0784] Step 10: Receiving User Input

[0785] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[0786] Specific operation: The device uses a microphone to capture the user's voice and processes the voice data in real time.

[0787] Input: Voice, text, and motion data from the user

[0788] Output: Captured input data

[0789] Step 11: Send Input

[0790] The device sends the captured user input data to the server using HTTPS.

[0791] Specific operation: The device encodes the captured audio data into BASE64 and sends it via an HTTPS request.

[0792] Input: Captured input data

[0793] Output: Input data sent to the server via secure communication

[0794] Step 12: Response Generation

[0795] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[0796] Specific operation: The server analyzes the user's input text and generates a response text based on its content using GPT-3.

[0797] Input: Captured user input data

[0798] Output: Generated response data

[0799] Step 13: Send response

[0800] The server sends the generated response to the terminal in an appropriate format, such as an audio file or text data.

[0801] Specific operation: The server converts the generated response text into audio using WaveNet and sends the audio file via HTTPS.

[0802] Input: Generated response data

[0803] Output: Response data sent to the terminal

[0804] Step 14: Present the response

[0805] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[0806] Specific operation: The terminal plays the requested audio data and lets the user listen to it.

[0807] Input: Response data received from the server

[0808] Output: The response presented to the user

[0809] (Application example 1)

[0810] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0811] Current digital security systems lack effective means to prevent unauthorized access. In particular, they do not adequately detect anomalies based on user behavior patterns, which increases the risk of unauthorized access and fraud. In addition, technology for managing user security using digital personas is not advanced, which prevents them from providing effective warnings and prevention measures.

[0812] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0813] In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from users and analyzing and responding in real time, and means for monitoring user behavior patterns to detect unauthorized access and issuing an alert when an abnormality is detected. This enables advanced anomaly detection based on user behavior patterns, making it possible to effectively prevent unauthorized access and fraud.

[0814] "Personal data" is information that can be associated with a specific individual, such as name, age, gender, voice, photograph, and behavioral patterns.

[0815] "Analyzing data and extracting individual characteristics" means using collected data and natural language processing and image recognition technology to reveal characteristics such as an individual's tone of voice and behavioral patterns.

[0816] A "generative artificial intelligence model" is an AI algorithm designed to perform specific tasks or responses based on collected data and analytical results.

[0817] A "digital personality" is a virtual personality formed by a generative artificial intelligence model that mimics the characteristics and behavioral patterns of a target individual.

[0818] "Multiple interfaces" refers to different platforms and devices, such as smartphones, PCs, AR / VR devices, and robots.

[0819] "User input" refers to various data that users provide to the system, such as voice, text, actions, and access logs.

[0820] "Analyzing and responding in real time" means instantly processing input from the user and generating and presenting an appropriate response.

[0821] "Monitoring user behavior patterns to detect unauthorized access" means analyzing user access logs and behavioral data to identify suspicious behavior that differs from normal behavior.

[0822] "Issuing a warning when an abnormality is detected" means that if the system detects any behavior that it deems to be fraudulent, it will immediately notify the user of the warning.

[0823] This invention relates to a security system that detects unauthorized access by collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement this invention.

[0824] User Data Collection Phase

[0825] Data upload

[0826] Users use smartphones or PCs to upload their own photos, voice messages, text data, and movement information through a dedicated application, and through this process, a variety of personal data is collected.

[0827] Data Conversion

[0828] The device locally converts the uploaded data into a standard format (JSON, XML, etc.) that is easy to parse and can be used in subsequent steps.

[0829] Data transmission

[0830] The converted data from the device is sent to the server using a secure protocol such as HTTPS, which ensures the privacy and security of the data.

[0831] Data analysis and learning phase

[0832] Data reception

[0833] The server receives the data sent from the terminal and temporarily stores it in storage.

[0834] Data analysis

[0835] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data and extract characteristics such as the user's tone of voice, vocabulary, and behavioral patterns.

[0836] Model learning

[0837] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, resulting in a digital personality model that reflects the user's characteristics.

[0838] Save Model

[0839] The server stores the learned digital personality model in storage and periodically updates it as needed.

[0840] Digital personality generation phase

[0841] Digital personality generation

[0842] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[0843] Interface Preparation

[0844] The server configures the generated digital personality to adapt to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots.

[0845] Interaction Phase

[0846] Receiving User Input

[0847] The device captures input from the user (voice, text, actions, etc.), and access logs and behavioral data are collected to detect unauthorized access.

[0848] Input Transmission

[0849] The terminal transmits the captured user input data to the server in real time.

[0850] Response Generation

[0851] The server analyzes the input it receives and generates an appropriate response, using a pre-trained model of the digital personality in the response generation process, along with algorithms designed to detect unauthorized access.

[0852] Response Send

[0853] The server generates a response and sends it to the terminal in the appropriate format.

[0854] Response suggestion

[0855] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[0856] Specific examples

[0857] Security system examples

[0858] 1. A user tries to unlock their smartphone.

[0859] 2. The device captures the user's operations and voice and sends the data to the server.

[0860] 3. The server analyzes the user's operation patterns and voice data to check for signs of unauthorized access. For example, if the user says, "This number is not mine."

[0861] 4. If the server detects an abnormality, it will send an alert to the terminal and display a response to the user saying "Security warning: Unauthorized access has been detected. Please check immediately."

[0862] This enables advanced anomaly detection based on user behavior patterns, effectively preventing unauthorized access and fraud.

[0863] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0864] Step 1:

[0865] Users upload personal data such as photos, voice messages, text data, and motion information from their smartphones or PCs through a dedicated application. The input data is diverse and includes, for example, the user's latest profile picture and recent conversation history. This data is converted into a standard format (such as JSON or XML) on the local device.

[0866] Step 2:

[0867] The device then sends the converted data to the server via a secure communication protocol (HTTPS), including the user's photos, voice messages, text data, and motion information, ensuring data privacy and security.

[0868] Step 3:

[0869] The server receives the data sent from the device and temporarily stores it in storage. The received data format is standard JSON or XML.

[0870] Step 4:

[0871] The server analyzes the received data and extracts characteristics such as the user's tone of voice, vocabulary, and behavioral patterns. These characteristics are analyzed using natural language processing (NLP) and image recognition technology. For example, NLP technology can be used to analyze the user's speaking style from a voice message, and image recognition technology can be used to analyze the user's facial expressions from a photograph.

[0872] Step 5:

[0873] The server trains a generative artificial intelligence model based on the extracted characteristics to generate a digital personality. This digital personality model reflects the user's characteristics and has the ability to respond via voice and text. The generated digital personality is stored in storage and updated periodically as needed.

[0874] Step 6:

[0875] The server configures the generated digital persona to adapt to multiple interfaces (e.g., smartphones, PCs, AR / VR devices, robots), providing a consistent digital persona across all platforms.

[0876] Step 7:

[0877] When a user tries to unlock their smartphone, the device captures the user's actions and voice. The input data includes, for example, the timing of the unlock operation and the voice message. The captured data is sent to the server in real time.

[0878] Step 8:

[0879] The server analyzes the received user input data and checks for signs of unauthorized access. It uses an AI model to analyze behavioral patterns and generates a warning if an anomaly is detected. For example, it is designed to detect an anomaly if a user says, "This number is not mine."

[0880] Step 9:

[0881] If the server detects an abnormality, it will send a warning to the device. This warning will be in the form of "Security Alert: Unauthorized access has been detected. Please check immediately."

[0882] Step 10:

[0883] The device will present the warning received from the server to the user by outputting audio in the case of voice, displaying on the screen in the case of text, and outputting both audio and video in the case of AR / VR devices and robots.

[0884] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0885] The present invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more human-like responses. The following embodiments explain how the present invention is specifically implemented.

[0886] User Data Collection Phase

[0887] Data upload

[0888] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[0889] Data Conversion

[0890] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[0891] Data transmission

[0892] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[0893] Data analysis and learning phase

[0894] Data reception

[0895] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[0896] Data analysis

[0897] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[0898] emotion recognition

[0899] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[0900] Model learning

[0901] The server then uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[0902] Save Model

[0903] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[0904] Digital personality generation phase

[0905] Digital personality generation

[0906] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[0907] Interface Preparation

[0908] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[0909] Interaction Phase

[0910] Receiving User Input

[0911] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[0912] Input Transmission

[0913] The device transmits the captured user input data to the server in real time, and may convert the voice data into text data before transmitting.

[0914] emotion recognition

[0915] The server analyzes the user's emotional state from the received input data and generates a response based on the analysis results.

[0916] Response Generation

[0917] The server analyzes the received input data and emotional information and generates optimal responses based on the user's words and actions, using a pre-trained digital personality model and emotion engine.

[0918] Response Send

[0919] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[0920] Response suggestion

[0921] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[0922] Specific examples

[0923] Specific examples of remote conversations

[0924] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[0925] 2. The device converts the speech into text and sends it to the server.

[0926] 3. The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[0927] 4. The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[0928] 5. The device converts the generated text into speech and conveys it to the user.

[0929] Specific examples of conversations using AR devices

[0930] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[0931] 2. The device captures the user's voice and movements and sends them to the server.

[0932] 3. The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[0933] 4. The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[0934] 5. The terminal presents the generated response to the user via audio and video.

[0935] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[0936] The processing flow will be explained below.

[0937] User Data Collection Phase

[0938] Step 1:

[0939] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[0940] Step 2:

[0941] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[0942] Step 3:

[0943] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[0944] Data analysis and learning phase

[0945] Step 4:

[0946] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[0947] Step 5:

[0948] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[0949] emotion recognition

[0950] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[0951] Step 6:

[0952] The server uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[0953] Step 7:

[0954] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[0955] Digital personality generation phase

[0956] Step 8:

[0957] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[0958] Step 9:

[0959] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[0960] Interaction Phase

[0961] Step 10:

[0962] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[0963] Step 11:

[0964] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[0965] Step 12:

[0966] The server analyzes the received input data and uses an emotion engine to recognize the user's emotional state. It combines emotional information with text data to understand the user's intentions.

[0967] Step 13:

[0968] The server generates appropriate responses based on the user's emotional information, using a pre-trained digital personality model to create natural responses that match the user's emotions.

[0969] Step 14:

[0970] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[0971] Step 15:

[0972] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[0973] Specific examples

[0974] Specific examples of remote conversations

[0975] Step 1:

[0976] The user uses their smartphone to say, "Dad, I had a hard time at work today."

[0977] Step 2:

[0978] The device converts the speech into text and sends it to the server.

[0979] Step 3:

[0980] The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[0981] Step 4:

[0982] The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[0983] Step 5:

[0984] The terminal converts the generated text into speech and conveys it to the user.

[0985] Specific examples of conversations using AR devices

[0986] Step 1:

[0987] The user puts on an AR headset and talks to a digital mom in their living room.

[0988] Step 2:

[0989] The device captures the user's voice and movements and sends them to a server.

[0990] Step 3:

[0991] The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[0992] Step 4:

[0993] The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[0994] Step 5:

[0995] The terminal presents the generated response to the user audio and visually.

[0996] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[0997] Example 2

[0998] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0999] In today's world, there is a growing demand for systems that provide responses that understand the user's characteristics and emotions. However, existing systems lack the ability to collect and analyze personal data, making it difficult to accurately recognize the user's emotions and generate responses. Furthermore, while there is a need for systems that can support a variety of interfaces, there is a lack of efficient means for achieving this. The purpose of this invention is to solve the above problems.

[1000] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for converting the collected data into a standard format, means for analyzing the converted data and extracting individual characteristics, means for recognizing an emotional state from the extracted characteristics, means for training a generative artificial intelligence model based on the characteristic information and emotional information and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, and means for capturing input from a user and analyzing and responding in real time. This makes it possible to provide responses that accurately reflect the user's characteristics and emotions through a variety of interfaces.

[1001] "Personal data" refers to information about a user, such as photos, videos, voice messages, and text data.

[1002] "Converting to a standard format" refers to converting collected data into a common format such as JSON or XML.

[1003] "Extracting personal characteristics" means analyzing characteristic information such as the user's tone of voice, choice of words, facial expressions, and behavioral patterns from the received data.

[1004] "Recognizing emotional state" means identifying the user's emotions (joy, sadness, anger, surprise, etc.) from the analyzed data.

[1005] A "generative artificial intelligence model" is an artificial intelligence model that learns using machine learning techniques such as neural networks and generates responses based on the user's characteristics and emotions.

[1006] A "digital persona" is a virtual personality that reproduces the user's characteristics and emotions, generated based on collected data and generative artificial intelligence models.

[1007] "Adapting to multiple interfaces" means adjusting the generated digital personality so that it can be used on different platforms, such as smartphones, PCs, AR / VR devices, and robots.

[1008] "Analyzing and responding in real time" means instantly analyzing input from the user and instantly generating and sending back an appropriate response based on that input.

[1009] MODE FOR CARRYING OUT THE INVENTION

[1010] This invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it aims to achieve more human-like responses.

[1011] User Data Collection Phase

[1012] Users upload their photos, videos, voice messages, text data, etc. to a specific application or web portal via their smartphone or PC. The device receives the uploaded data and converts it locally into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[1013] Data transmission and analysis

[1014] The device sends the converted data to the server. A secure communication protocol such as HTTPS is used for transmission, ensuring data safety. The server receives the data sent from the device and temporarily stores it in storage. At this stage, verification is performed to confirm the data's integrity and consistency.

[1015] The server then analyzes the received data to extract characteristic information such as the user's tone of voice, vocabulary, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) techniques and image recognition algorithms. The server then uses an emotion engine to recognize the user's emotional state from the information obtained through data analysis. For example, it can determine whether the user is happy, sad, angry, or surprised.

[1016] Model training and saving

[1017] The server uses a generative artificial intelligence model to learn the user's digital personality based on the extracted characteristic information and emotional information. Deep learning technology is used to train the neural network. The learned digital personality model is stored in storage and is periodically updated with new data, ensuring that the digital personality reflects the latest information.

[1018] Digital personality generation and interface adaptation

[1019] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The generated digital personality faithfully reproduces the user's characteristics and emotions. The server also configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[1020] Interaction Phase

[1021] Users can interact with their digital persona through the device. The device captures input from the user (voice, text, actions, etc.) and sends it to the server in real time. The server then uses the received input data to analyze the user's emotional state using an emotion engine, and generates a response based on the analysis results. The generated response is sent from the server to the device and presented to the user.

[1022] Specific examples

[1023] Specific examples of remote conversations

[1024] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[1025] 2. The device converts the voice into text using automatic speech recognition (ASR) technology and sends it to the server.

[1026] 3. The server analyzes the received text using NLP algorithms and uses an emotion engine to recognize the user's emotions, such as stress or fatigue.

[1027] 4. The server generates a response based on the user's emotional information and past data: "That must have been tough. What happened?"

[1028] 5. The device converts the generated text into speech using text-to-speech (TTS) technology and outputs it through the speaker.

[1029] Specific examples of conversations using AR devices

[1030] 1. A user puts on an AR headset and talks to a virtual digital personality of their mother in their living room.

[1031] 2. The device captures the user's voice and movements using a camera and sensors and sends them to the server.

[1032] 3. The server analyzes the user's comments and actions using NLP and an emotion engine and determines that the user is happy.

[1033] 4. The server generates a response saying, "Wow, that's a nice photo. Where did you take it?"

[1034] 5. The device uses audio and AR images to present the response to the user.

[1035] The system allows users to interact naturally with their digital persona, maintaining an emotional connection through emotionally aware responses. Examples of prompts include "I'm so tired today. Tell me something uplifting," "Let's talk about what happened with your friend yesterday," and "Mom, what do you think about the news these days?"

[1036] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1037] Step 1: Upload data

[1038] Users use their smartphones or PCs to upload personal data such as photos, videos, voice messages, and text data to specific applications or web portals. These data are input, and the device receives the data.

[1039] Step 2: Data conversion

[1040] The terminal converts the uploaded data it receives into a standard format (e.g., JSON, XML). This conversion process includes changing the format of the data and adding metadata. The input raw data is reformatted, and the output is data in a standard format.

[1041] Step 3: Send data

[1042] The device sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure the data is secure. The input is data in a standard format, and the output is sent to the server.

[1043] Step 4: Data reception and storage

[1044] The server receives data sent from the device and temporarily stores it in storage. It verifies the received data to ensure its integrity and consistency. It also verifies the input data and outputs it as stored data.

[1045] Step 5: Data analysis

[1046] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns. The analysis uses natural language processing (NLP) technology and image recognition algorithms. The input data is analyzed and the extracted characteristic information is output.

[1047] Step 6: Emotion Recognition

[1048] The server uses an emotion engine based on the characteristic information obtained from the data analysis to recognize the user's emotional state, for example, joy, sadness, anger, surprise, etc. The input is the analyzed characteristic information, and the output is the recognized emotion information.

[1049] Step 7: Model training

[1050] The server uses a generative artificial intelligence model to train a digital personality based on the extracted trait and emotion information. This process uses deep learning techniques to train a neural network. The input is the trait and emotion information, and the output is the trained model.

[1051] Step 8: Save the model

[1052] The server saves the trained digital personality model in storage, and periodically updates the model with new data. The saved model is obtained as output.

[1053] Step 9: Digital Persona Generation

[1054] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The input data is the user's real-time information, and the output is a response.

[1055] Step 10: Interface Adaptation

[1056] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converts data formats, and generates output appropriate for each interface.

[1057] Step 11: Receiving User Input

[1058] The device captures input from the user (voice, text, movement, etc.). When using an AR device or robot as an action, the input is obtained using a camera or sensor. The captured data becomes the input.

[1059] Step 12: Send Input

[1060] The device sends the captured user input data to the server in real time. In the case of voice data, it is converted to text using ASR technology before being sent. The input is the captured data, and the data sent to the server is the output.

[1061] Step 13: Response Generation

[1062] The server uses the digital personality to generate the optimal response based on the input data and emotional information it receives. Analysis and response generation are performed based on the input data, and the generated response is obtained as output.

[1063] Step 14: Send response

[1064] The server converts the generated response into an appropriate format (audio file, text data, etc.) and sends it to the device. An optimized response is generated as output and sent to the device.

[1065] Step 15: Present the response

[1066] The device presents the response received from the server to the user. Voice responses are output from the speaker, and text responses are displayed on the screen. AR / VR devices use audio and video for output. The presented response is the final output.

[1067] (Application example 2)

[1068] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1069] Conventional customer service systems have difficulty in responding to customer emotions and personal characteristics, making it difficult to achieve human-like dialogue. This has made it difficult to improve customer satisfaction in brick-and-mortar stores. It has also been difficult to provide appropriate product introductions and personalized services.

[1070] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from customers and analyzing and responding in real time, means for recognizing emotions and generating appropriate responses, and means for providing customer service. This enables more human-like interactions and service provision that reflects the customer's emotions and personal characteristics.

[1071] "Personal data" refers to information such as an individual's photos, videos, voice messages, and text data.

[1072] "Means of collection" refers to the methods and devices used to obtain data using devices such as smartphones and PCs.

[1073] "Means of analysis" refers to the technology used to analyze the acquired data and extract individual characteristics and behavioral patterns.

[1074] "Personal characteristics" refer to an individual's unique characteristics, such as tone of voice, language, facial expressions, and behavioral patterns.

[1075] A "generative artificial intelligence model" refers to an artificial intelligence that learns based on collected data and has specific functions and capabilities.

[1076] "Digital personas" refer to digital agents created using generative artificial intelligence models that replicate human characteristics and emotions.

[1077] "Means of adaptation" refers to technology that allows the generated digital personality to be compatible with multiple interfaces (smartphones, PCs, AR / VR devices, etc.).

[1078] "Means for capturing input and analyzing and responding in real time" refers to technology that receives user voice or text input, analyzes it immediately, and generates an appropriate response.

[1079] "Means for performing emotion recognition and generating appropriate responses" refers to technology that detects the user's emotional state and generates an optimal response based on that emotion.

[1080] "Means for providing customer service" refers to methods and technologies for providing product information and personalized services to customers in physical stores.

[1081] "Customer" refers to consumers or users who visit a physical store.

[1082] This invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more human-like responses.

[1083] System configuration and program processing

[1084] Hardware and Software:

[1085] Hardware: Smartphones, smart glasses, PCs, servers

[1086] Software: Python, TensorFlow, DeepFace, Transformers (Hugging Face)

[1087] The server includes means for collecting and analyzing personal data, means for generating a digital personality using a generative artificial intelligence model, means for adapting to multiple interfaces, means for capturing input from customers and analyzing and responding in real time, means for performing emotion recognition, and means for providing customer service.

[1088] Program processing procedure

[1089] Data collection:

[1090] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, and more to an application or web portal.

[1091] Data conversion and transmission:

[1092] The device converts the uploaded data into a standard format (e.g., JSON, XML) and sends it to the server using a secure communication protocol, encrypting the data before sending it.

[1093] Data analysis and model training:

[1094] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) technology and image recognition algorithms. Based on the extracted characteristics and emotional information, a generative AI model is trained to generate a digital personality.

[1095] Digital personality and emotion recognition:

[1096] The server combines the generated digital personality with an emotion engine to recognize the user's emotions in real time and generate corresponding responses, using TensorFlow and DeepFace for this process.

[1097] Interface Adaptation:

[1098] The generated digital personality is adapted to multiple interfaces such as smartphones, smart glasses, and PCs.

[1099] Specific examples

[1100] Example 1: Customer service via smartphone

[1101] 1. A user uses their smartphone to ask, "What are the features of this product?"

[1102] 2. The device converts the speech into text and sends it to the server.

[1103] 3. The server analyzes the text and uses an emotion engine to determine which content the user is interested in.

[1104] 4. The server generates the appropriate response: "This product has a modern design and is particularly durable."

[1105] 5. The device converts the generated response into speech and conveys it to the user.

[1106] Example 2: Customer service using smart glasses

[1107] 1. The user asks through the smart glasses, "How do I use this product?"

[1108] 2. The device captures audio and movement and sends it to the server.

[1109] 3. The server analyzes the user's question and uses an emotion engine to recognize their interest.

[1110] 4. The server generates the appropriate response: "Using this product is very easy. First..."

[1111] 5. The terminal presents the generated response to the user via audio and video.

[1112] Example prompt sentence:

[1113] Customer question: "What are the features of this product?"

[1114] Emotion: "Excited"

[1115] Generate responses for your digital persona.

[1116] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1117] Step 1: Data collection

[1118] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to an application or web portal. The input data becomes the raw material for analyzing and extracting personal characteristics.

[1119] Step 2: Data conversion

[1120] The device receives the uploaded data, converts it into a standard format (e.g., JSON, XML), adds metadata, and conforms to the format, and then transmits the converted data to the server using a secure communication protocol (e.g., HTTPS).

[1121] Step 3: Data reception and analysis

[1122] The server receives the converted data sent from the device and temporarily stores it in storage. Based on the input data, it uses natural language processing (NLP) technology and image recognition algorithms to analyze and extract characteristics such as the user's tone of voice, vocabulary, facial expressions, and behavioral patterns.

[1123] Step 4: Emotion Recognition

[1124] The server uses the information obtained from the data analysis to analyze the user's emotional state using an emotion engine, identifying the type of emotion (e.g., joy, sadness, anger, etc.), and further processing the data based on the analysis results.

[1125] Step 5: Model training and digital personality generation

[1126] The server trains a generative AI model based on the extracted characteristics and emotional information to generate a digital personality. The training process uses neural network technology (e.g., TensorFlow) to train the model. The generated digital personality model is then stored in storage.

[1127] Step 6: Interface Adaptation

[1128] The server adapts the generated digital personality to multiple interfaces (smartphones, smart glasses, PCs, etc.), optimizing its response and display formats for each device.

[1129] Step 7: Receiving User Input

[1130] When a user sends a question or request through a smartphone or smart glasses, the device captures the input (voice, text, movement, etc.) and converts the voice data to text as needed before transmitting the input to the server in real time.

[1131] Step 8: Response Generation

[1132] The server analyzes the user's input data and emotional information and generates an appropriate response. This process again uses generative AI models and an emotion engine. The appropriate response is converted into text or voice.

[1133] Step 9: Send and present the response

[1134] The server converts the generated response into an appropriate format (audio file, text data) and sends it to the device, which then presents the received response to the user. In the case of smartphones or smart glasses, the response is provided as audio or displayed on a screen.

[1135] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1136] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1137] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1138] [Third embodiment]

[1139] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1140] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1141] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1142] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1143] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1144] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1145] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1146] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1147] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1148] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1149] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1150] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1151] The present invention relates to a system for collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement the present invention.

[1152] User Data Collection Phase

[1153] Data upload

[1154] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. This process is carried out through a dedicated application or web portal.

[1155] Data Conversion

[1156] The device processes the uploaded data locally and converts it into a standard format (JSON, XML, etc.) that is then formatted in a way that is suitable for subsequent analysis steps.

[1157] Data transmission

[1158] The device then sends the converted data to a server using a secure communication protocol such as HTTPS to ensure privacy and data security.

[1159] Data analysis and learning phase

[1160] Data reception

[1161] The server receives the data sent from the device, temporarily stores it in storage, and uses it for the analysis process.

[1162] Data analysis

[1163] The server analyzes the received data and extracts characteristics such as the user's tone of voice, language, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) and image recognition technologies.

[1164] Model learning

[1165] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, which then generates a digital personality that reflects the user's characteristics.

[1166] Save Model

[1167] The server stores the trained digital personality model in storage and periodically updates it, ensuring that the digital personality reflects the latest data.

[1168] Digital personality generation phase

[1169] Digital personality generation

[1170] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[1171] Interface Preparation

[1172] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[1173] Interaction Phase

[1174] Receiving User Input

[1175] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[1176] Input Transmission

[1177] The terminal transmits the captured user input data to the server in real time.

[1178] Response Generation

[1179] The server analyzes the received input and generates an appropriate response, using a pre-trained digital personality model in the response generation process.

[1180] Response Send

[1181] The server generates a response and sends it to the device in an appropriate format (e.g., an audio file or text data).

[1182] Response suggestion

[1183] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[1184] Specific examples

[1185] Specific examples of remote conversations

[1186] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[1187] 2. The device converts the speech into text and sends it to the server.

[1188] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[1189] 4. The device converts the generated text into speech and conveys it to the user.

[1190] Specific examples of conversations using AR devices

[1191] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[1192] 2. The device captures the user's voice and movements and sends them to the server.

[1193] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[1194] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[1195] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[1196] The processing flow will be explained below.

[1197] User Data Collection Phase

[1198] Step 1:

[1199] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[1200] Step 2:

[1201] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[1202] Step 3:

[1203] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[1204] Data analysis and learning phase

[1205] Step 4:

[1206] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[1207] Step 5:

[1208] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[1209] Step 6:

[1210] The server uses the extracted characteristics information to train a user's digital personality using a generative artificial intelligence model. The training process uses a neural network and involves repeated training.

[1211] Step 7:

[1212] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[1213] Digital personality generation phase

[1214] Step 8:

[1215] The server uses the user's data and a trained AI model to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics.

[1216] Step 9:

[1217] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[1218] Interaction Phase

[1219] Step 10:

[1220] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[1221] Step 11:

[1222] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[1223] Step 12:

[1224] The server analyzes the received input data and generates an optimal response based on the user's words and actions, using a pre-trained digital personality model.

[1225] Step 13:

[1226] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[1227] Step 14:

[1228] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[1229] This specific processing flow allows the system to receive input from the user in real time and realize natural interactions with the digital personality that responds appropriately.

[1230] Example 1

[1231] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1232] While modern technology exists to collect and analyze personal data to generate digital personalities, the efficiency and security of the data conversion, transmission, and analysis processes may not be sufficiently ensured. Furthermore, the generated digital personalities have limited flexibility in adapting to multiple interfaces and limited ability to respond to user input in real time. This not only results in a poor user experience, but also poses privacy risks.

[1233] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1234] In this invention, the server includes: means for collecting personal data; means for converting the collected data into a standard format (JSON, XML); means for transmitting the converted data using a secure communication protocol (HTTPS); means for receiving the transmitted data and temporarily storing it in storage; means for analyzing the stored data and extracting personal characteristics using natural language processing or image recognition; means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality; means for storing the generated digital personality in storage and periodically updating it; means for adapting the generated digital personality to multiple interfaces; means for capturing input from a user, analyzing it in real time, and generating a response; means for transmitting the generated response to a terminal and presenting it to the user; and means for transmitting the generated response to the terminal and presenting it to the user. This enables efficient and secure data conversion and transmission, enables the generated digital personality to flexibly adapt to multiple interfaces, and enables immediate and appropriate responses to real-time user input.

[1235] "Personal Data" refers to information about a user, such as a user's photos, videos, voice messages, text data, etc.

[1236] "Standard formats" refer to general-purpose data formats such as JSON and XML that are used to make data compatible between different systems.

[1237] A "secure communication protocol" is a protocol that ensures the confidentiality and integrity of data when sending and receiving information, and examples include HTTPS.

[1238] "Storage" refers to physical or virtual storage devices for storing data, including cloud storage and local disk drives.

[1239] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language; examples include speech recognition and text analysis.

[1240] "Image recognition technology" is a technology that recognizes and analyzes patterns and features in images, and examples include facial recognition and object detection.

[1241] A "generative artificial intelligence model" is an artificial intelligence model that generates new data and responses based on user data, and an example is GPT-3.

[1242] A "digital persona" is a computer-generated virtual personality that reflects the user's characteristics and can interact with the user through voice, text, video, etc.

[1243] An "interface" is a means or device that allows a user to interact with a system, and includes smartphones, PCs, AR / VR devices, robots, etc.

[1244] "User input" refers to data such as voice, text, and actions that a user provides to the system.

[1245] "Real-time analysis" refers to the process of instantly analyzing input from a user and generating a response without delay.

[1246] A "response" is a reply or reaction that a system generates in response to input from a user, and may be in the form of audio, text, or video.

[1247] The present invention is a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. This system enables natural dialogue based on the user's data and maintains a psychological connection. The following describes how the present invention is specifically implemented.

[1248] User Data Collection Phase

[1249] Data upload

[1250] Users upload their own photos, videos, voice messages, and text data using dedicated applications or web portals. Specific hardware used includes smartphones (e.g., iPhone, Android) and PCs (e.g., Windows PC, Mac). Software used includes dedicated data upload applications (e.g., digital personality apps) and web portals.

[1251] Data Conversion

[1252] The terminal receives the uploaded data and converts it into a standard format (JSON, XML). This process uses Python and its libraries (e.g., Pandas, json library). For example, a JPEG photo uploaded by a user is BASE64 encoded and converted into JSON format.

[1253] Data transmission

[1254] The terminal transmits the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[1255] Data analysis and learning phase

[1256] Data reception

[1257] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[1258] Data analysis

[1259] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.). This analysis uses natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TensorFlow).

[1260] Model learning

[1261] The server then trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user, using frameworks such as PyTorch and TensorFlow.

[1262] Save Model

[1263] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[1264] Digital personality generation phase

[1265] Digital personality generation

[1266] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[1267] Interface Preparation

[1268] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[1269] Interaction Phase

[1270] Receiving User Input

[1271] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[1272] Input Transmission

[1273] The device sends the captured user input data to the server using HTTPS.

[1274] Response Generation

[1275] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[1276] Response Send

[1277] The server sends the generated response to the terminal in an appropriate format, for example, as an audio file or text data.

[1278] Response suggestion

[1279] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[1280] Specific examples

[1281] Specific examples of remote conversations

[1282] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[1283] 2. The device converts the speech into text and sends it to the server.

[1284] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[1285] 4. The device converts the generated text into speech and conveys it to the user.

[1286] Specific examples of conversations using AR devices

[1287] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[1288] 2. The device captures the user's voice and movements and sends them to the server.

[1289] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[1290] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[1291] Example prompt sentence:

[1292] "Dad, how was the meeting today?"

[1293] "Mom, what did you have for lunch today?"

[1294] "How was your day? Anything new?"

[1295] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[1296] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1297] Step 1: Upload data

[1298] Users can upload their photos, videos, voice messages, and text data using a dedicated application or web portal. Specifically, they open the application, click the "Upload Data" button, select a file, and then press the "Send" button.

[1299] Input: Personal data on your smartphone or PC (photos, videos, voice messages, text data)

[1300] Output: Uploaded personal data is sent to the device.

[1301] Step 2: Data conversion

[1302] The terminal receives the uploaded data and converts it into a standard format (JSON, XML) using Python and its libraries (e.g., Pandas, json library).

[1303] Specific operation: For example, the terminal encodes a JPEG image using BASE64 and converts it into JSON format.

[1304] Input: Uploaded raw data (JPEG, MP4, WAV, TXT, etc.)

[1305] Output: Data converted to a standard format (BASE64 encoded JSON file)

[1306] Step 3: Send data

[1307] The terminal sends the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[1308] Specific operation: Include the converted data in the payload portion of the HTTPS request and post it to the server's API endpoint.

[1309] Input: Data converted to a standard format (BASE64 encoded JSON file)

[1310] Output: Data sent to the server via secure communication

[1311] Step 4: Receiving Data

[1312] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[1313] Specific operation: The server receives the HTTPS request, analyzes the payload portion, and executes instructions to save it in storage.

[1314] Input: Data received via HTTPS

[1315] Output: Data saved in storage

[1316] Step 5: Data analysis

[1317] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.) using natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TensorFlow).

[1318] Specific operation: The server converts, for example, an audio file into text, and analyzes the text to extract emotions and keywords.

[1319] Input: Data stored in storage

[1320] Output: User characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.)

[1321] Step 6: Model training

[1322] The server trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user. Frameworks such as PyTorch and TensorFlow are used.

[1323] Specific operation: The server inputs the user's characteristic data into the model and optimizes the model parameters.

[1324] Input: User characteristics data

[1325] Output: A trained generative artificial intelligence model

[1326] Step 7: Save the model

[1327] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[1328] Specific operation: The server periodically loads the trained model, retrains it with new data, and overwrites the updated model in the same storage.

[1329] Input: A trained generative artificial intelligence model

[1330] Output: The trained model saved in storage.

[1331] Step 8: Digital Persona Generation

[1332] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[1333] Specific operation: The server uses GPT-3 to generate phrases that the user might say, and then converts that text into speech using WaveNet.

[1334] Input: trained generative artificial intelligence model and user characteristics

[1335] Output: Response data such as audio, text, or video

[1336] Step 9: Prepare the interface

[1337] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[1338] Specific operation: The server uses the API of each device to convert voice data and text data into a format that can be distributed.

[1339] Input: Voice and text data

[1340] Output: Data adapted to each device

[1341] Step 10: Receiving User Input

[1342] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[1343] Specific operation: The device uses a microphone to capture the user's voice and processes the voice data in real time.

[1344] Input: Voice, text, and motion data from the user

[1345] Output: Captured input data

[1346] Step 11: Send Input

[1347] The device sends the captured user input data to the server using HTTPS.

[1348] Specific operation: The device encodes the captured audio data into BASE64 and sends it via an HTTPS request.

[1349] Input: Captured input data

[1350] Output: Input data sent to the server via secure communication

[1351] Step 12: Response Generation

[1352] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[1353] Specific operation: The server analyzes the user's input text and generates a response text based on its content using GPT-3.

[1354] Input: Captured user input data

[1355] Output: Generated response data

[1356] Step 13: Send response

[1357] The server sends the generated response to the terminal in an appropriate format, such as an audio file or text data.

[1358] Specific operation: The server converts the generated response text into audio using WaveNet and sends the audio file via HTTPS.

[1359] Input: Generated response data

[1360] Output: Response data sent to the terminal

[1361] Step 14: Present the response

[1362] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[1363] Specific operation: The terminal plays the requested audio data and lets the user listen to it.

[1364] Input: Response data received from the server

[1365] Output: The response presented to the user

[1366] (Application example 1)

[1367] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1368] Current digital security systems lack effective means to prevent unauthorized access. In particular, they do not adequately detect anomalies based on user behavior patterns, which increases the risk of unauthorized access and fraud. In addition, technology for managing user security using digital personas is not advanced, which prevents them from providing effective warnings and prevention measures.

[1369] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1370] In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from users and analyzing and responding in real time, and means for monitoring user behavior patterns to detect unauthorized access and issuing an alert when an abnormality is detected. This enables advanced anomaly detection based on user behavior patterns, making it possible to effectively prevent unauthorized access and fraud.

[1371] "Personal data" is information that can be associated with a specific individual, such as name, age, gender, voice, photograph, and behavioral patterns.

[1372] "Analyzing data and extracting individual characteristics" means using collected data and natural language processing and image recognition technology to reveal characteristics such as an individual's tone of voice and behavioral patterns.

[1373] A "generative artificial intelligence model" is an AI algorithm designed to perform specific tasks or responses based on collected data and analytical results.

[1374] A "digital personality" is a virtual personality formed by a generative artificial intelligence model that mimics the characteristics and behavioral patterns of a target individual.

[1375] "Multiple interfaces" refers to different platforms and devices, such as smartphones, PCs, AR / VR devices, and robots.

[1376] "User input" refers to various data that users provide to the system, such as voice, text, actions, and access logs.

[1377] "Analyzing and responding in real time" means instantly processing input from the user and generating and presenting an appropriate response.

[1378] "Monitoring user behavior patterns to detect unauthorized access" means analyzing user access logs and behavioral data to identify suspicious behavior that differs from normal behavior.

[1379] "Issuing a warning when an abnormality is detected" means that if the system detects any behavior that it deems to be fraudulent, it will immediately notify the user of the warning.

[1380] This invention relates to a security system that detects unauthorized access by collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement this invention.

[1381] User Data Collection Phase

[1382] Data upload

[1383] Users use smartphones or PCs to upload their own photos, voice messages, text data, and movement information through a dedicated application, and through this process, a variety of personal data is collected.

[1384] Data Conversion

[1385] The device locally converts the uploaded data into a standard format (JSON, XML, etc.) that is easy to parse and can be used in subsequent steps.

[1386] Data transmission

[1387] The converted data from the device is sent to the server using a secure protocol such as HTTPS, which ensures the privacy and security of the data.

[1388] Data analysis and learning phase

[1389] Data reception

[1390] The server receives the data sent from the terminal and temporarily stores it in storage.

[1391] Data analysis

[1392] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data and extract characteristics such as the user's tone of voice, vocabulary, and behavioral patterns.

[1393] Model learning

[1394] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, resulting in a digital personality model that reflects the user's characteristics.

[1395] Save Model

[1396] The server stores the learned digital personality model in storage and periodically updates it as needed.

[1397] Digital personality generation phase

[1398] Digital personality generation

[1399] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[1400] Interface Preparation

[1401] The server configures the generated digital personality to adapt to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots.

[1402] Interaction Phase

[1403] Receiving User Input

[1404] The device captures input from the user (voice, text, actions, etc.), and access logs and behavioral data are collected to detect unauthorized access.

[1405] Input Transmission

[1406] The terminal transmits the captured user input data to the server in real time.

[1407] Response Generation

[1408] The server analyzes the input it receives and generates an appropriate response, using a pre-trained model of the digital personality in the response generation process, along with algorithms designed to detect unauthorized access.

[1409] Response Send

[1410] The server generates a response and sends it to the terminal in the appropriate format.

[1411] Response suggestion

[1412] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[1413] Specific examples

[1414] Security system examples

[1415] 1. A user tries to unlock their smartphone.

[1416] 2. The device captures the user's operations and voice and sends the data to the server.

[1417] 3. The server analyzes the user's operation patterns and voice data to check for signs of unauthorized access. For example, if the user says, "This number is not mine."

[1418] 4. If the server detects an abnormality, it will send an alert to the terminal and display a response to the user saying "Security warning: Unauthorized access has been detected. Please check immediately."

[1419] This enables advanced anomaly detection based on user behavior patterns, effectively preventing unauthorized access and fraud.

[1420] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1421] Step 1:

[1422] Users upload personal data such as photos, voice messages, text data, and motion information from their smartphones or PCs through a dedicated application. The input data is diverse and includes, for example, the user's latest profile picture and recent conversation history. This data is converted into a standard format (such as JSON or XML) on the local device.

[1423] Step 2:

[1424] The device then sends the converted data to the server via a secure communication protocol (HTTPS), including the user's photos, voice messages, text data, and motion information, ensuring data privacy and security.

[1425] Step 3:

[1426] The server receives the data sent from the device and temporarily stores it in storage. The received data format is standard JSON or XML.

[1427] Step 4:

[1428] The server analyzes the received data and extracts characteristics such as the user's tone of voice, vocabulary, and behavioral patterns. These characteristics are analyzed using natural language processing (NLP) and image recognition technology. For example, NLP technology can be used to analyze the user's speaking style from a voice message, and image recognition technology can be used to analyze the user's facial expressions from a photograph.

[1429] Step 5:

[1430] The server trains a generative artificial intelligence model based on the extracted characteristics to generate a digital personality. This digital personality model reflects the user's characteristics and has the ability to respond via voice and text. The generated digital personality is stored in storage and updated periodically as needed.

[1431] Step 6:

[1432] The server configures the generated digital persona to adapt to multiple interfaces (e.g., smartphones, PCs, AR / VR devices, robots), providing a consistent digital persona across all platforms.

[1433] Step 7:

[1434] When a user tries to unlock their smartphone, the device captures the user's actions and voice. The input data includes, for example, the timing of the unlock operation and the voice message. The captured data is sent to the server in real time.

[1435] Step 8:

[1436] The server analyzes the received user input data and checks for signs of unauthorized access. It uses an AI model to analyze behavioral patterns and generates a warning if an anomaly is detected. For example, it is designed to detect an anomaly if a user says, "This number is not mine."

[1437] Step 9:

[1438] If the server detects an abnormality, it will send a warning to the device. This warning will be in the form of "Security Alert: Unauthorized access has been detected. Please check immediately."

[1439] Step 10:

[1440] The device will present the warning received from the server to the user by outputting audio in the case of voice, displaying on the screen in the case of text, and outputting both audio and video in the case of AR / VR devices and robots.

[1441] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1442] The present invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more human-like responses. The following embodiments explain how the present invention is specifically implemented.

[1443] User Data Collection Phase

[1444] Data upload

[1445] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[1446] Data Conversion

[1447] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[1448] Data transmission

[1449] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[1450] Data analysis and learning phase

[1451] Data reception

[1452] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[1453] Data analysis

[1454] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[1455] emotion recognition

[1456] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[1457] Model learning

[1458] The server then uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[1459] Save Model

[1460] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[1461] Digital personality generation phase

[1462] Digital personality generation

[1463] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[1464] Interface Preparation

[1465] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[1466] Interaction Phase

[1467] Receiving User Input

[1468] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[1469] Input Transmission

[1470] The device transmits the captured user input data to the server in real time, and may convert the voice data into text data before transmitting.

[1471] emotion recognition

[1472] The server analyzes the user's emotional state from the received input data and generates a response based on the analysis results.

[1473] Response Generation

[1474] The server analyzes the received input data and emotional information and generates optimal responses based on the user's words and actions, using a pre-trained digital personality model and emotion engine.

[1475] Response Send

[1476] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[1477] Response suggestion

[1478] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[1479] Specific examples

[1480] Specific examples of remote conversations

[1481] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[1482] 2. The device converts the speech into text and sends it to the server.

[1483] 3. The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[1484] 4. The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[1485] 5. The device converts the generated text into speech and conveys it to the user.

[1486] Specific examples of conversations using AR devices

[1487] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[1488] 2. The device captures the user's voice and movements and sends them to the server.

[1489] 3. The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[1490] 4. The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[1491] 5. The terminal presents the generated response to the user via audio and video.

[1492] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[1493] The processing flow will be explained below.

[1494] User Data Collection Phase

[1495] Step 1:

[1496] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[1497] Step 2:

[1498] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[1499] Step 3:

[1500] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[1501] Data analysis and learning phase

[1502] Step 4:

[1503] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[1504] Step 5:

[1505] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[1506] emotion recognition

[1507] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[1508] Step 6:

[1509] The server uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[1510] Step 7:

[1511] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[1512] Digital personality generation phase

[1513] Step 8:

[1514] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[1515] Step 9:

[1516] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[1517] Interaction Phase

[1518] Step 10:

[1519] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[1520] Step 11:

[1521] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[1522] Step 12:

[1523] The server analyzes the received input data and uses an emotion engine to recognize the user's emotional state. It combines emotional information with text data to understand the user's intentions.

[1524] Step 13:

[1525] The server generates appropriate responses based on the user's emotional information, using a pre-trained digital personality model to create natural responses that match the user's emotions.

[1526] Step 14:

[1527] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[1528] Step 15:

[1529] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[1530] Specific examples

[1531] Specific examples of remote conversations

[1532] Step 1:

[1533] The user uses their smartphone to say, "Dad, I had a hard time at work today."

[1534] Step 2:

[1535] The device converts the speech into text and sends it to the server.

[1536] Step 3:

[1537] The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[1538] Step 4:

[1539] The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[1540] Step 5:

[1541] The terminal converts the generated text into speech and conveys it to the user.

[1542] Specific examples of conversations using AR devices

[1543] Step 1:

[1544] The user puts on an AR headset and talks to a digital mom in their living room.

[1545] Step 2:

[1546] The device captures the user's voice and movements and sends them to a server.

[1547] Step 3:

[1548] The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[1549] Step 4:

[1550] The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[1551] Step 5:

[1552] The terminal presents the generated response to the user audio and visually.

[1553] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[1554] Example 2

[1555] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1556] In today's world, there is a growing demand for systems that provide responses that understand the user's characteristics and emotions. However, existing systems lack the ability to collect and analyze personal data, making it difficult to accurately recognize the user's emotions and generate responses. Furthermore, while there is a need for systems that can support a variety of interfaces, there is a lack of efficient means for achieving this. The purpose of this invention is to solve the above problems.

[1557] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for converting the collected data into a standard format, means for analyzing the converted data and extracting individual characteristics, means for recognizing an emotional state from the extracted characteristics, means for training a generative artificial intelligence model based on the characteristic information and emotional information and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, and means for capturing input from a user and analyzing and responding in real time. This makes it possible to provide responses that accurately reflect the user's characteristics and emotions through a variety of interfaces.

[1558] "Personal data" refers to information about a user, such as photos, videos, voice messages, and text data.

[1559] "Converting to a standard format" refers to converting collected data into a common format such as JSON or XML.

[1560] "Extracting personal characteristics" means analyzing characteristic information such as the user's tone of voice, choice of words, facial expressions, and behavioral patterns from the received data.

[1561] "Recognizing emotional state" means identifying the user's emotions (joy, sadness, anger, surprise, etc.) from the analyzed data.

[1562] A "generative artificial intelligence model" is an artificial intelligence model that learns using machine learning techniques such as neural networks and generates responses based on the user's characteristics and emotions.

[1563] A "digital persona" is a virtual personality that reproduces the user's characteristics and emotions, generated based on collected data and generative artificial intelligence models.

[1564] "Adapting to multiple interfaces" means adjusting the generated digital personality so that it can be used on different platforms, such as smartphones, PCs, AR / VR devices, and robots.

[1565] "Analyzing and responding in real time" means instantly analyzing input from the user and instantly generating and sending back an appropriate response based on that input.

[1566] MODE FOR CARRYING OUT THE INVENTION

[1567] This invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it aims to achieve more human-like responses.

[1568] User Data Collection Phase

[1569] Users upload their photos, videos, voice messages, text data, etc. to a specific application or web portal via their smartphone or PC. The device receives the uploaded data and converts it locally into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[1570] Data transmission and analysis

[1571] The device sends the converted data to the server. A secure communication protocol such as HTTPS is used for transmission, ensuring data safety. The server receives the data sent from the device and temporarily stores it in storage. At this stage, verification is performed to confirm the data's integrity and consistency.

[1572] The server then analyzes the received data to extract characteristic information such as the user's tone of voice, vocabulary, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) techniques and image recognition algorithms. The server then uses an emotion engine to recognize the user's emotional state from the information obtained through data analysis. For example, it can determine whether the user is happy, sad, angry, or surprised.

[1573] Model training and saving

[1574] The server uses a generative artificial intelligence model to learn the user's digital personality based on the extracted characteristic information and emotional information. Deep learning technology is used to train the neural network. The learned digital personality model is stored in storage and is periodically updated with new data, ensuring that the digital personality reflects the latest information.

[1575] Digital personality generation and interface adaptation

[1576] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The generated digital personality faithfully reproduces the user's characteristics and emotions. The server also configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[1577] Interaction Phase

[1578] Users can interact with their digital persona through the device. The device captures input from the user (voice, text, actions, etc.) and sends it to the server in real time. The server then uses the received input data to analyze the user's emotional state using an emotion engine, and generates a response based on the analysis results. The generated response is sent from the server to the device and presented to the user.

[1579] Specific examples

[1580] Specific examples of remote conversations

[1581] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[1582] 2. The device converts the voice into text using automatic speech recognition (ASR) technology and sends it to the server.

[1583] 3. The server analyzes the received text using NLP algorithms and uses an emotion engine to recognize the user's emotions, such as stress or fatigue.

[1584] 4. The server generates a response based on the user's emotional information and past data: "That must have been tough. What happened?"

[1585] 5. The device converts the generated text into speech using text-to-speech (TTS) technology and outputs it through the speaker.

[1586] Specific examples of conversations using AR devices

[1587] 1. A user puts on an AR headset and talks to a virtual digital personality of their mother in their living room.

[1588] 2. The device captures the user's voice and movements using a camera and sensors and sends them to the server.

[1589] 3. The server analyzes the user's comments and actions using NLP and an emotion engine and determines that the user is happy.

[1590] 4. The server generates a response saying, "Wow, that's a nice photo. Where did you take it?"

[1591] 5. The device uses audio and AR images to present the response to the user.

[1592] The system allows users to interact naturally with their digital persona, maintaining an emotional connection through emotionally aware responses. Examples of prompts include "I'm so tired today. Tell me something uplifting," "Let's talk about what happened with your friend yesterday," and "Mom, what do you think about the news these days?"

[1593] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1594] Step 1: Upload data

[1595] Users use their smartphones or PCs to upload personal data such as photos, videos, voice messages, and text data to specific applications or web portals. These data are input, and the device receives the data.

[1596] Step 2: Data conversion

[1597] The terminal converts the uploaded data it receives into a standard format (e.g., JSON, XML). This conversion process includes changing the format of the data and adding metadata. The input raw data is reformatted, and the output is data in a standard format.

[1598] Step 3: Send data

[1599] The device sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure the data is secure. The input is data in a standard format, and the output is sent to the server.

[1600] Step 4: Data reception and storage

[1601] The server receives data sent from the device and temporarily stores it in storage. It verifies the received data to ensure its integrity and consistency. It also verifies the input data and outputs it as stored data.

[1602] Step 5: Data analysis

[1603] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns. The analysis uses natural language processing (NLP) technology and image recognition algorithms. The input data is analyzed and the extracted characteristic information is output.

[1604] Step 6: Emotion Recognition

[1605] The server uses an emotion engine based on the characteristic information obtained from the data analysis to recognize the user's emotional state, for example, joy, sadness, anger, surprise, etc. The input is the analyzed characteristic information, and the output is the recognized emotion information.

[1606] Step 7: Model training

[1607] The server uses a generative artificial intelligence model to train a digital personality based on the extracted trait and emotion information. This process uses deep learning techniques to train a neural network. The input is the trait and emotion information, and the output is the trained model.

[1608] Step 8: Save the model

[1609] The server saves the trained digital personality model in storage, and periodically updates the model with new data. The saved model is obtained as output.

[1610] Step 9: Digital Persona Generation

[1611] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The input data is the user's real-time information, and the output is a response.

[1612] Step 10: Interface Adaptation

[1613] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converts data formats, and generates output appropriate for each interface.

[1614] Step 11: Receiving User Input

[1615] The device captures input from the user (voice, text, movement, etc.). When using an AR device or robot as an action, the input is obtained using a camera or sensor. The captured data becomes the input.

[1616] Step 12: Send Input

[1617] The device sends the captured user input data to the server in real time. In the case of voice data, it is converted to text using ASR technology before being sent. The input is the captured data, and the data sent to the server is the output.

[1618] Step 13: Response Generation

[1619] The server uses the digital personality to generate the optimal response based on the input data and emotional information it receives. Analysis and response generation are performed based on the input data, and the generated response is obtained as output.

[1620] Step 14: Send response

[1621] The server converts the generated response into an appropriate format (audio file, text data, etc.) and sends it to the device. An optimized response is generated as output and sent to the device.

[1622] Step 15: Present the response

[1623] The device presents the response received from the server to the user. Voice responses are output from the speaker, and text responses are displayed on the screen. AR / VR devices use audio and video for output. The presented response is the final output.

[1624] (Application example 2)

[1625] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1626] Conventional customer service systems have difficulty in responding to customer emotions and personal characteristics, making it difficult to achieve human-like dialogue. This has made it difficult to improve customer satisfaction in brick-and-mortar stores. It has also been difficult to provide appropriate product introductions and personalized services.

[1627] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from customers and analyzing and responding in real time, means for recognizing emotions and generating appropriate responses, and means for providing customer service. This enables more human-like interactions and service provision that reflects the customer's emotions and personal characteristics.

[1628] "Personal data" refers to information such as an individual's photos, videos, voice messages, and text data.

[1629] "Means of collection" refers to the methods and devices used to obtain data using devices such as smartphones and PCs.

[1630] "Means of analysis" refers to the technology used to analyze the acquired data and extract individual characteristics and behavioral patterns.

[1631] "Personal characteristics" refer to an individual's unique characteristics, such as tone of voice, language, facial expressions, and behavioral patterns.

[1632] A "generative artificial intelligence model" refers to an artificial intelligence that learns based on collected data and has specific functions and capabilities.

[1633] "Digital personas" refer to digital agents created using generative artificial intelligence models that replicate human characteristics and emotions.

[1634] "Means of adaptation" refers to technology that allows the generated digital personality to be compatible with multiple interfaces (smartphones, PCs, AR / VR devices, etc.).

[1635] "Means for capturing input and analyzing and responding in real time" refers to technology that receives user voice or text input, analyzes it immediately, and generates an appropriate response.

[1636] "Means for performing emotion recognition and generating appropriate responses" refers to technology that detects the user's emotional state and generates an optimal response based on that emotion.

[1637] "Means for providing customer service" refers to methods and technologies for providing product information and personalized services to customers in physical stores.

[1638] "Customer" refers to consumers or users who visit a physical store.

[1639] This invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more human-like responses.

[1640] System configuration and program processing

[1641] Hardware and Software:

[1642] Hardware: Smartphones, smart glasses, PCs, servers

[1643] Software: Python, TensorFlow, DeepFace, Transformers (Hugging Face)

[1644] The server includes means for collecting and analyzing personal data, means for generating a digital personality using a generative artificial intelligence model, means for adapting to multiple interfaces, means for capturing input from customers and analyzing and responding in real time, means for performing emotion recognition, and means for providing customer service.

[1645] Program processing procedure

[1646] Data collection:

[1647] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, and more to an application or web portal.

[1648] Data conversion and transmission:

[1649] The device converts the uploaded data into a standard format (e.g., JSON, XML) and sends it to the server using a secure communication protocol, encrypting the data before sending it.

[1650] Data analysis and model training:

[1651] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) technology and image recognition algorithms. Based on the extracted characteristics and emotional information, a generative AI model is trained to generate a digital personality.

[1652] Digital personality and emotion recognition:

[1653] The server combines the generated digital personality with an emotion engine to recognize the user's emotions in real time and generate corresponding responses, using TensorFlow and DeepFace for this process.

[1654] Interface Adaptation:

[1655] The generated digital personality is adapted to multiple interfaces such as smartphones, smart glasses, and PCs.

[1656] Specific examples

[1657] Example 1: Customer service via smartphone

[1658] 1. A user uses their smartphone to ask, "What are the features of this product?"

[1659] 2. The device converts the speech into text and sends it to the server.

[1660] 3. The server analyzes the text and uses an emotion engine to determine which content the user is interested in.

[1661] 4. The server generates the appropriate response: "This product has a modern design and is particularly durable."

[1662] 5. The device converts the generated response into speech and conveys it to the user.

[1663] Example 2: Customer service using smart glasses

[1664] 1. The user asks through the smart glasses, "How do I use this product?"

[1665] 2. The device captures audio and movement and sends it to the server.

[1666] 3. The server analyzes the user's question and uses an emotion engine to recognize their interest.

[1667] 4. The server generates the appropriate response: "Using this product is very easy. First..."

[1668] 5. The terminal presents the generated response to the user via audio and video.

[1669] Example prompt sentence:

[1670] Customer question: "What are the features of this product?"

[1671] Emotion: "Excited"

[1672] Generate responses for your digital persona.

[1673] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1674] Step 1: Data collection

[1675] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to an application or web portal. The input data becomes the raw material for analyzing and extracting personal characteristics.

[1676] Step 2: Data conversion

[1677] The device receives the uploaded data, converts it into a standard format (e.g., JSON, XML), adds metadata, and conforms to the format, and then transmits the converted data to the server using a secure communication protocol (e.g., HTTPS).

[1678] Step 3: Data reception and analysis

[1679] The server receives the converted data sent from the device and temporarily stores it in storage. Based on the input data, it uses natural language processing (NLP) technology and image recognition algorithms to analyze and extract characteristics such as the user's tone of voice, vocabulary, facial expressions, and behavioral patterns.

[1680] Step 4: Emotion Recognition

[1681] The server uses the information obtained from the data analysis to analyze the user's emotional state using an emotion engine, identifying the type of emotion (e.g., joy, sadness, anger, etc.), and further processing the data based on the analysis results.

[1682] Step 5: Model training and digital personality generation

[1683] The server trains a generative AI model based on the extracted characteristics and emotional information to generate a digital personality. The training process uses neural network technology (e.g., TensorFlow) to train the model. The generated digital personality model is then stored in storage.

[1684] Step 6: Interface Adaptation

[1685] The server adapts the generated digital personality to multiple interfaces (smartphones, smart glasses, PCs, etc.), optimizing its response and display formats for each device.

[1686] Step 7: Receiving User Input

[1687] When a user sends a question or request through a smartphone or smart glasses, the device captures the input (voice, text, movement, etc.) and converts the voice data to text as needed before transmitting the input to the server in real time.

[1688] Step 8: Response Generation

[1689] The server analyzes the user's input data and emotional information and generates an appropriate response. This process again uses generative AI models and an emotion engine. The appropriate response is converted into text or voice.

[1690] Step 9: Send and present the response

[1691] The server converts the generated response into an appropriate format (audio file, text data) and sends it to the device, which then presents the received response to the user. In the case of smartphones or smart glasses, the response is provided as audio or displayed on a screen.

[1692] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1693] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1694] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1695] [Fourth embodiment]

[1696] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1697] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1698] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1699] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1700] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1701] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1702] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1703] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1704] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1705] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1706] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1707] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1708] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1709] The present invention relates to a system for collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement the present invention.

[1710] User Data Collection Phase

[1711] Data upload

[1712] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. This process is carried out through a dedicated application or web portal.

[1713] Data Conversion

[1714] The device processes the uploaded data locally and converts it into a standard format (JSON, XML, etc.) that is then formatted in a way that is suitable for subsequent analysis steps.

[1715] Data transmission

[1716] The device then sends the converted data to a server using a secure communication protocol such as HTTPS to ensure privacy and data security.

[1717] Data analysis and learning phase

[1718] Data reception

[1719] The server receives the data sent from the device, temporarily stores it in storage, and uses it for the analysis process.

[1720] Data analysis

[1721] The server analyzes the received data and extracts characteristics such as the user's tone of voice, language, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) and image recognition technologies.

[1722] Model learning

[1723] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, which then generates a digital personality that reflects the user's characteristics.

[1724] Save Model

[1725] The server stores the trained digital personality model in storage and periodically updates it, ensuring that the digital personality reflects the latest data.

[1726] Digital personality generation phase

[1727] Digital personality generation

[1728] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[1729] Interface Preparation

[1730] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[1731] Interaction Phase

[1732] Receiving User Input

[1733] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[1734] Input Transmission

[1735] The terminal transmits the captured user input data to the server in real time.

[1736] Response Generation

[1737] The server analyzes the received input and generates an appropriate response, using a pre-trained digital personality model in the response generation process.

[1738] Response Send

[1739] The server generates a response and sends it to the device in an appropriate format (e.g., an audio file or text data).

[1740] Response suggestion

[1741] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[1742] Specific examples

[1743] Specific examples of remote conversations

[1744] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[1745] 2. The device converts the speech into text and sends it to the server.

[1746] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[1747] 4. The device converts the generated text into speech and conveys it to the user.

[1748] Specific examples of conversations using AR devices

[1749] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[1750] 2. The device captures the user's voice and movements and sends them to the server.

[1751] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[1752] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[1753] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[1754] The processing flow will be explained below.

[1755] User Data Collection Phase

[1756] Step 1:

[1757] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[1758] Step 2:

[1759] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[1760] Step 3:

[1761] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[1762] Data analysis and learning phase

[1763] Step 4:

[1764] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[1765] Step 5:

[1766] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[1767] Step 6:

[1768] The server uses the extracted characteristics information to train a user's digital personality using a generative artificial intelligence model. The training process uses a neural network and involves repeated training.

[1769] Step 7:

[1770] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[1771] Digital personality generation phase

[1772] Step 8:

[1773] The server uses the user's data and a trained AI model to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics.

[1774] Step 9:

[1775] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[1776] Interaction Phase

[1777] Step 10:

[1778] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[1779] Step 11:

[1780] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[1781] Step 12:

[1782] The server analyzes the received input data and generates an optimal response based on the user's words and actions, using a pre-trained digital personality model.

[1783] Step 13:

[1784] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[1785] Step 14:

[1786] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[1787] This specific processing flow allows the system to receive input from the user in real time and realize natural interactions with the digital personality that responds appropriately.

[1788] Example 1

[1789] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1790] While modern technology exists to collect and analyze personal data to generate digital personalities, the efficiency and security of the data conversion, transmission, and analysis processes may not be sufficiently ensured. Furthermore, the generated digital personalities have limited flexibility in adapting to multiple interfaces and limited ability to respond to user input in real time. This not only results in a poor user experience, but also poses privacy risks.

[1791] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1792] In this invention, the server includes: means for collecting personal data; means for converting the collected data into a standard format (JSON, XML); means for transmitting the converted data using a secure communication protocol (HTTPS); means for receiving the transmitted data and temporarily storing it in storage; means for analyzing the stored data and extracting personal characteristics using natural language processing or image recognition; means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality; means for storing the generated digital personality in storage and periodically updating it; means for adapting the generated digital personality to multiple interfaces; means for capturing input from a user, analyzing it in real time, and generating a response; means for transmitting the generated response to a terminal and presenting it to the user; and means for transmitting the generated response to the terminal and presenting it to the user. This enables efficient and secure data conversion and transmission, enables the generated digital personality to flexibly adapt to multiple interfaces, and enables immediate and appropriate responses to real-time user input.

[1793] "Personal Data" refers to information about a user, such as a user's photos, videos, voice messages, text data, etc.

[1794] "Standard formats" refer to general-purpose data formats such as JSON and XML that are used to make data compatible between different systems.

[1795] A "secure communication protocol" is a protocol that ensures the confidentiality and integrity of data when sending and receiving information, and examples include HTTPS.

[1796] "Storage" refers to physical or virtual storage devices for storing data, including cloud storage and local disk drives.

[1797] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language; examples include speech recognition and text analysis.

[1798] "Image recognition technology" is a technology that recognizes and analyzes patterns and features in images, and examples include facial recognition and object detection.

[1799] A "generative artificial intelligence model" is an artificial intelligence model that generates new data and responses based on user data, and an example is GPT-3.

[1800] A "digital persona" is a computer-generated virtual personality that reflects the user's characteristics and can interact with the user through voice, text, video, etc.

[1801] An "interface" is a means or device that allows a user to interact with a system, and includes smartphones, PCs, AR / VR devices, robots, etc.

[1802] "User input" refers to data such as voice, text, and actions that a user provides to the system.

[1803] "Real-time analysis" refers to the process of instantly analyzing input from a user and generating a response without delay.

[1804] A "response" is a reply or reaction that a system generates in response to input from a user, and may be in the form of audio, text, or video.

[1805] The present invention is a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. This system enables natural dialogue based on the user's data and maintains a psychological connection. The following describes how the present invention is specifically implemented.

[1806] User Data Collection Phase

[1807] Data upload

[1808] Users upload their own photos, videos, voice messages, and text data using dedicated applications or web portals. Specific hardware used includes smartphones (e.g., iPhone, Android) and PCs (e.g., Windows PC, Mac). Software used includes dedicated data upload applications (e.g., digital personality apps) and web portals.

[1809] Data Conversion

[1810] The terminal receives the uploaded data and converts it into a standard format (JSON, XML). This process uses Python and its libraries (e.g., Pandas, json library). For example, a JPEG photo uploaded by a user is BASE64 encoded and converted into JSON format.

[1811] Data transmission

[1812] The terminal transmits the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[1813] Data analysis and learning phase

[1814] Data reception

[1815] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[1816] Data analysis

[1817] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.). This analysis uses natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TensorFlow).

[1818] Model learning

[1819] The server then trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user, using frameworks such as PyTorch and TensorFlow.

[1820] Save Model

[1821] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[1822] Digital personality generation phase

[1823] Digital personality generation

[1824] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[1825] Interface Preparation

[1826] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[1827] Interaction Phase

[1828] Receiving User Input

[1829] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[1830] Input Transmission

[1831] The device sends the captured user input data to the server using HTTPS.

[1832] Response Generation

[1833] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[1834] Response Send

[1835] The server sends the generated response to the terminal in an appropriate format, for example, as an audio file or text data.

[1836] Response suggestion

[1837] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[1838] Specific examples

[1839] Specific examples of remote conversations

[1840] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[1841] 2. The device converts the speech into text and sends it to the server.

[1842] 3. The server parses the received text and generates an appropriate response based on past data, such as "That was tough. What happened?"

[1843] 4. The device converts the generated text into speech and conveys it to the user.

[1844] Specific examples of conversations using AR devices

[1845] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[1846] 2. The device captures the user's voice and movements and sends them to the server.

[1847] 3. The server generates an appropriate response based on the user's words and actions. For example, a user might say, "Mom, look at this," and point to a photo they're holding.

[1848] 4. The device displays and speaks the generated response (e.g., "Wow, that's a nice photo. Where did you take it?") to the user.

[1849] Example prompt sentence:

[1850] "Dad, how was the meeting today?"

[1851] "Mom, what did you have for lunch today?"

[1852] "How was your day? Anything new?"

[1853] The system allows users to naturally interact with the deceased or their own digital personas, maintaining a spiritual connection.

[1854] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1855] Step 1: Upload data

[1856] Users can upload their photos, videos, voice messages, and text data using a dedicated application or web portal. Specifically, they open the application, click the "Upload Data" button, select a file, and then press the "Send" button.

[1857] Input: Personal data on your smartphone or PC (photos, videos, voice messages, text data)

[1858] Output: Uploaded personal data is sent to the device.

[1859] Step 2: Data conversion

[1860] The terminal receives the uploaded data and converts it into a standard format (JSON, XML) using Python and its libraries (e.g., Pandas, json library).

[1861] Specific operation: For example, the terminal encodes a JPEG image using BASE64 and converts it into JSON format.

[1862] Input: Uploaded raw data (JPEG, MP4, WAV, TXT, etc.)

[1863] Output: Data converted to a standard format (BASE64 encoded JSON file)

[1864] Step 3: Send data

[1865] The terminal sends the converted data to the server using a secure communication protocol (HTTPS). Encryption technology is used to ensure security.

[1866] Specific operation: Include the converted data in the payload portion of the HTTPS request and post it to the server's API endpoint.

[1867] Input: Data converted to a standard format (BASE64 encoded JSON file)

[1868] Output: Data sent to the server via secure communication

[1869] Step 4: Receiving Data

[1870] The server receives the data sent from the device and temporarily stores it in storage such as an Amazon S3 bucket.

[1871] Specific operation: The server receives the HTTPS request, analyzes the payload portion, and executes instructions to save it in storage.

[1872] Input: Data received via HTTPS

[1873] Output: Data saved in storage

[1874] Step 5: Data analysis

[1875] The server analyzes the stored data and extracts user characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.) using natural language processing technology (e.g., spaCy, BERT) and image recognition technology (e.g., OpenCV, TensorFlow).

[1876] Specific operation: The server converts, for example, an audio file into text, and analyzes the text to extract emotions and keywords.

[1877] Input: Data stored in storage

[1878] Output: User characteristics (tone of voice, language, facial expressions, behavioral patterns, etc.)

[1879] Step 6: Model training

[1880] The server trains a generative artificial intelligence model (e.g., GPT-3) based on the extracted characteristics to generate a digital personality for the user. Frameworks such as PyTorch and TensorFlow are used.

[1881] Specific operation: The server inputs the user's characteristic data into the model and optimizes the model parameters.

[1882] Input: User characteristics data

[1883] Output: A trained generative artificial intelligence model

[1884] Step 7: Save the model

[1885] The server stores the trained digital personality model in storage such as Amazon S3 or Google Cloud Storage and periodically updates it with new data.

[1886] Specific operation: The server periodically loads the trained model, retrains it with new data, and overwrites the updated model in the same storage.

[1887] Input: A trained generative artificial intelligence model

[1888] Output: The trained model saved in storage.

[1889] Step 8: Digital Persona Generation

[1890] The server generates a digital personality based on the trained AI model, which responds with voice, text, and video that reflects the user's characteristics. For example, the generated text is converted into voice using WaveNet.

[1891] Specific operation: The server uses GPT-3 to generate phrases that the user might say, and then converts that text into speech using WaveNet.

[1892] Input: trained generative artificial intelligence model and user characteristics

[1893] Output: Response data such as audio, text, or video

[1894] Step 9: Prepare the interface

[1895] The server adapts the generated digital personality to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots, using each device's API and SDK.

[1896] Specific operation: The server uses the API of each device to convert voice data and text data into a format that can be distributed.

[1897] Input: Voice and text data

[1898] Output: Data adapted to each device

[1899] Step 10: Receiving User Input

[1900] The device captures input from the user (voice, text, actions, etc.) using a microphone, camera, sensors, etc.

[1901] Specific operation: The device uses a microphone to capture the user's voice and processes the voice data in real time.

[1902] Input: Voice, text, and motion data from the user

[1903] Output: Captured input data

[1904] Step 11: Send Input

[1905] The device sends the captured user input data to the server using HTTPS.

[1906] Specific operation: The device encodes the captured audio data into BASE64 and sends it via an HTTPS request.

[1907] Input: Captured input data

[1908] Output: Input data sent to the server via secure communication

[1909] Step 12: Response Generation

[1910] The server analyzes user input in real time and uses generative artificial intelligence models to generate appropriate responses.

[1911] Specific operation: The server analyzes the user's input text and generates a response text based on its content using GPT-3.

[1912] Input: Captured user input data

[1913] Output: Generated response data

[1914] Step 13: Send response

[1915] The server sends the generated response to the terminal in an appropriate format, such as an audio file or text data.

[1916] Specific operation: The server converts the generated response text into audio using WaveNet and sends the audio file via HTTPS.

[1917] Input: Generated response data

[1918] Output: Response data sent to the terminal

[1919] Step 14: Present the response

[1920] The device then presents the response received from the server to the user. For example, on a smartphone, voice responses are sent via the speaker and text responses are displayed on the screen. On AR / VR devices and robots, both audio and video are used.

[1921] Specific operation: The terminal plays the requested audio data and lets the user listen to it.

[1922] Input: Response data received from the server

[1923] Output: The response presented to the user

[1924] (Application example 1)

[1925] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1926] Current digital security systems lack effective means to prevent unauthorized access. In particular, they do not adequately detect anomalies based on user behavior patterns, which increases the risk of unauthorized access and fraud. In addition, technology for managing user security using digital personas is not advanced, which prevents them from providing effective warnings and prevention measures.

[1927] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1928] In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from users and analyzing and responding in real time, and means for monitoring user behavior patterns to detect unauthorized access and issuing an alert when an abnormality is detected. This enables advanced anomaly detection based on user behavior patterns, making it possible to effectively prevent unauthorized access and fraud.

[1929] "Personal data" is information that can be associated with a specific individual, such as name, age, gender, voice, photograph, and behavioral patterns.

[1930] "Analyzing data and extracting individual characteristics" means using collected data and natural language processing and image recognition technology to reveal characteristics such as an individual's tone of voice and behavioral patterns.

[1931] A "generative artificial intelligence model" is an AI algorithm designed to perform specific tasks or responses based on collected data and analytical results.

[1932] A "digital personality" is a virtual personality formed by a generative artificial intelligence model that mimics the characteristics and behavioral patterns of a target individual.

[1933] "Multiple interfaces" refers to different platforms and devices, such as smartphones, PCs, AR / VR devices, and robots.

[1934] "User input" refers to various data that users provide to the system, such as voice, text, actions, and access logs.

[1935] "Analyzing and responding in real time" means instantly processing input from the user and generating and presenting an appropriate response.

[1936] "Monitoring user behavior patterns to detect unauthorized access" means analyzing user access logs and behavioral data to identify suspicious behavior that differs from normal behavior.

[1937] "Issuing a warning when an abnormality is detected" means that if the system detects any behavior that it deems to be fraudulent, it will immediately notify the user of the warning.

[1938] This invention relates to a security system that detects unauthorized access by collecting and analyzing personal data and generating a digital personality using a generative artificial intelligence model. The following embodiments explain how to specifically implement this invention.

[1939] User Data Collection Phase

[1940] Data upload

[1941] Users use smartphones or PCs to upload their own photos, voice messages, text data, and movement information through a dedicated application, and through this process, a variety of personal data is collected.

[1942] Data Conversion

[1943] The device locally converts the uploaded data into a standard format (JSON, XML, etc.) that is easy to parse and can be used in subsequent steps.

[1944] Data transmission

[1945] The converted data from the device is sent to the server using a secure protocol such as HTTPS, which ensures the privacy and security of the data.

[1946] Data analysis and learning phase

[1947] Data reception

[1948] The server receives the data sent from the terminal and temporarily stores it in storage.

[1949] Data analysis

[1950] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data and extract characteristics such as the user's tone of voice, vocabulary, and behavioral patterns.

[1951] Model learning

[1952] The server uses the extracted information to learn how to generate a digital personality for the user using a generative artificial intelligence model, resulting in a digital personality model that reflects the user's characteristics.

[1953] Save Model

[1954] The server stores the learned digital personality model in storage and periodically updates it as needed.

[1955] Digital personality generation phase

[1956] Digital personality generation

[1957] The server generates a digital personality that responds via voice, text, video, etc. based on the user's data and a trained AI model.

[1958] Interface Preparation

[1959] The server configures the generated digital personality to adapt to multiple interfaces, such as smartphones, PCs, AR / VR devices, and robots.

[1960] Interaction Phase

[1961] Receiving User Input

[1962] The device captures input from the user (voice, text, actions, etc.), and access logs and behavioral data are collected to detect unauthorized access.

[1963] Input Transmission

[1964] The terminal transmits the captured user input data to the server in real time.

[1965] Response Generation

[1966] The server analyzes the input it receives and generates an appropriate response, using a pre-trained model of the digital personality in the response generation process, along with algorithms designed to detect unauthorized access.

[1967] Response Send

[1968] The server generates a response and sends it to the terminal in the appropriate format.

[1969] Response suggestion

[1970] The device presents the response received from the server to the user. For voice responses, it outputs voice, for text responses it displays on the screen, and for AR / VR devices and robots, it outputs both voice and video.

[1971] Specific examples

[1972] Security system examples

[1973] 1. A user tries to unlock their smartphone.

[1974] 2. The device captures the user's operations and voice and sends the data to the server.

[1975] 3. The server analyzes the user's operation patterns and voice data to check for signs of unauthorized access. For example, if the user says, "This number is not mine."

[1976] 4. If the server detects an abnormality, it will send an alert to the terminal and display a response to the user saying "Security warning: Unauthorized access has been detected. Please check immediately."

[1977] This enables advanced anomaly detection based on user behavior patterns, effectively preventing unauthorized access and fraud.

[1978] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1979] Step 1:

[1980] Users upload personal data such as photos, voice messages, text data, and motion information from their smartphones or PCs through a dedicated application. The input data is diverse and includes, for example, the user's latest profile picture and recent conversation history. This data is converted into a standard format (such as JSON or XML) on the local device.

[1981] Step 2:

[1982] The device then sends the converted data to the server via a secure communication protocol (HTTPS), including the user's photos, voice messages, text data, and motion information, ensuring data privacy and security.

[1983] Step 3:

[1984] The server receives the data sent from the device and temporarily stores it in storage. The received data format is standard JSON or XML.

[1985] Step 4:

[1986] The server analyzes the received data and extracts characteristics such as the user's tone of voice, vocabulary, and behavioral patterns. These characteristics are analyzed using natural language processing (NLP) and image recognition technology. For example, NLP technology can be used to analyze the user's speaking style from a voice message, and image recognition technology can be used to analyze the user's facial expressions from a photograph.

[1987] Step 5:

[1988] The server trains a generative artificial intelligence model based on the extracted characteristics to generate a digital personality. This digital personality model reflects the user's characteristics and has the ability to respond via voice and text. The generated digital personality is stored in storage and updated periodically as needed.

[1989] Step 6:

[1990] The server configures the generated digital persona to adapt to multiple interfaces (e.g., smartphones, PCs, AR / VR devices, robots), providing a consistent digital persona across all platforms.

[1991] Step 7:

[1992] When a user tries to unlock their smartphone, the device captures the user's actions and voice. The input data includes, for example, the timing of the unlock operation and the voice message. The captured data is sent to the server in real time.

[1993] Step 8:

[1994] The server analyzes the received user input data and checks for signs of unauthorized access. It uses an AI model to analyze behavioral patterns and generates a warning if an anomaly is detected. For example, it is designed to detect an anomaly if a user says, "This number is not mine."

[1995] Step 9:

[1996] If the server detects an abnormality, it will send a warning to the device. This warning will be in the form of "Security Alert: Unauthorized access has been detected. Please check immediately."

[1997] Step 10:

[1998] The device will present the warning received from the server to the user by outputting audio in the case of voice, displaying on the screen in the case of text, and outputting both audio and video in the case of AR / VR devices and robots.

[1999] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2000] The present invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system achieves more human-like responses. The following embodiments explain how the present invention is specifically implemented.

[2001] User Data Collection Phase

[2002] Data upload

[2003] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[2004] Data Conversion

[2005] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[2006] Data transmission

[2007] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[2008] Data analysis and learning phase

[2009] Data reception

[2010] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[2011] Data analysis

[2012] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[2013] emotion recognition

[2014] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[2015] Model learning

[2016] The server then uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[2017] Save Model

[2018] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[2019] Digital personality generation phase

[2020] Digital personality generation

[2021] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[2022] Interface Preparation

[2023] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[2024] Interaction Phase

[2025] Receiving User Input

[2026] The device captures input from the user (voice, text, movement, etc.) In the case of AR devices and robots, this is done using cameras and sensors.

[2027] Input Transmission

[2028] The device transmits the captured user input data to the server in real time, and may convert the voice data into text data before transmitting.

[2029] emotion recognition

[2030] The server analyzes the user's emotional state from the received input data and generates a response based on the analysis results.

[2031] Response Generation

[2032] The server analyzes the received input data and emotional information and generates optimal responses based on the user's words and actions, using a pre-trained digital personality model and emotion engine.

[2033] Response Send

[2034] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[2035] Response suggestion

[2036] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[2037] Specific examples

[2038] Specific examples of remote conversations

[2039] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[2040] 2. The device converts the speech into text and sends it to the server.

[2041] 3. The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[2042] 4. The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[2043] 5. The device converts the generated text into speech and conveys it to the user.

[2044] Specific examples of conversations using AR devices

[2045] 1. A user puts on an AR headset and talks to their mother's digital personality in their living room.

[2046] 2. The device captures the user's voice and movements and sends them to the server.

[2047] 3. The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[2048] 4. The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[2049] 5. The terminal presents the generated response to the user via audio and video.

[2050] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[2051] The processing flow will be explained below.

[2052] User Data Collection Phase

[2053] Step 1:

[2054] Users use their smartphones or PCs to upload their photos, videos, voice messages, text data, etc. to a specific application or web portal.

[2055] Step 2:

[2056] The device receives the uploaded data and locally converts it into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[2057] Step 3:

[2058] The device then sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure data security.

[2059] Data analysis and learning phase

[2060] Step 4:

[2061] The server receives the data sent from the device and temporarily stores it in storage, where it performs verification to ensure the data is complete and consistent.

[2062] Step 5:

[2063] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns, using natural language processing (NLP) technology and image recognition algorithms.

[2064] emotion recognition

[2065] The server further analyzes the information obtained in the data analysis step using an emotion engine to recognize the user's emotional state (e.g., joy, sadness, anger, surprise, etc.) This emotion information is used to generate subsequent responses.

[2066] Step 6:

[2067] The server uses a generative AI model to learn the user's digital personality based on the extracted characteristics and emotions. The learning process uses a neural network and repeats training.

[2068] Step 7:

[2069] The server stores the trained digital personality model in storage and periodically updates it with new data, ensuring that the digital personality reflects the latest information.

[2070] Digital personality generation phase

[2071] Step 8:

[2072] The server uses user data, a trained AI model, and an emotion engine to generate a digital personality that responds in real time and faithfully reproduces the user's characteristics and emotions.

[2073] Step 9:

[2074] The server configures the generated digital personality to adapt to different interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converting it into an output format specific to each interface.

[2075] Interaction Phase

[2076] Step 10:

[2077] A user initiates a dialogue with a digital persona using a device such as a smartphone, PC, AR / VR device, or robot. For example, the user speaks into the smartphone.

[2078] Step 11:

[2079] The device captures the user's voice and text input and transmits the data to a server in real time, sometimes converting the voice data to text before transmitting it.

[2080] Step 12:

[2081] The server analyzes the received input data and uses an emotion engine to recognize the user's emotional state. It combines emotional information with text data to understand the user's intentions.

[2082] Step 13:

[2083] The server generates appropriate responses based on the user's emotional information, using a pre-trained digital personality model to create natural responses that match the user's emotions.

[2084] Step 14:

[2085] The server converts the generated response into an appropriate format (e.g., audio file, text data) and sends it to the device, where the response data is optimized for the interface.

[2086] Step 15:

[2087] The device presents the response received from the server to the user. Voice responses are output through the speaker, text responses are displayed on the screen, and AR / VR devices and robots output both audio and video.

[2088] Specific examples

[2089] Specific examples of remote conversations

[2090] Step 1:

[2091] The user uses their smartphone to say, "Dad, I had a hard time at work today."

[2092] Step 2:

[2093] The device converts the speech into text and sends it to the server.

[2094] Step 3:

[2095] The server analyzes the received text and uses an emotion engine to recognize the user's emotional state, for example, determining whether they are feeling stressed or tired.

[2096] Step 4:

[2097] The server generates an appropriate response based on the user's emotional information and past data: "That must have been tough. What happened?"

[2098] Step 5:

[2099] The terminal converts the generated text into speech and conveys it to the user.

[2100] Specific examples of conversations using AR devices

[2101] Step 1:

[2102] The user puts on an AR headset and talks to a digital mom in their living room.

[2103] Step 2:

[2104] The device captures the user's voice and movements and sends them to a server.

[2105] Step 3:

[2106] The server uses an emotion engine to recognize the user's emotional state based on their words and actions. For example, it determines that the user is happy.

[2107] Step 4:

[2108] The server generates an appropriate response based on the emotional information (e.g., "Wow, that's a nice photo. Where did you take it?").

[2109] Step 5:

[2110] The terminal presents the generated response to the user audio and visually.

[2111] The system allows users to naturally interact with the deceased or their own digital personas, and maintain a spiritual connection through emotionally aware responses.

[2112] Example 2

[2113] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2114] In today's world, there is a growing demand for systems that provide responses that understand the user's characteristics and emotions. However, existing systems lack the ability to collect and analyze personal data, making it difficult to accurately recognize the user's emotions and generate responses. Furthermore, while there is a need for systems that can support a variety of interfaces, there is a lack of efficient means for achieving this. The purpose of this invention is to solve the above problems.

[2115] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for converting the collected data into a standard format, means for analyzing the converted data and extracting individual characteristics, means for recognizing an emotional state from the extracted characteristics, means for training a generative artificial intelligence model based on the characteristic information and emotional information and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, and means for capturing input from a user and analyzing and responding in real time. This makes it possible to provide responses that accurately reflect the user's characteristics and emotions through a variety of interfaces.

[2116] "Personal data" refers to information about a user, such as photos, videos, voice messages, and text data.

[2117] "Converting to a standard format" refers to converting collected data into a common format such as JSON or XML.

[2118] "Extracting personal characteristics" means analyzing characteristic information such as the user's tone of voice, choice of words, facial expressions, and behavioral patterns from the received data.

[2119] "Recognizing emotional state" means identifying the user's emotions (joy, sadness, anger, surprise, etc.) from the analyzed data.

[2120] A "generative artificial intelligence model" is an artificial intelligence model that learns using machine learning techniques such as neural networks and generates responses based on the user's characteristics and emotions.

[2121] A "digital persona" is a virtual personality that reproduces the user's characteristics and emotions, generated based on collected data and generative artificial intelligence models.

[2122] "Adapting to multiple interfaces" means adjusting the generated digital personality so that it can be used on different platforms, such as smartphones, PCs, AR / VR devices, and robots.

[2123] "Analyzing and responding in real time" means instantly analyzing input from the user and instantly generating and sending back an appropriate response based on that input.

[2124] MODE FOR CARRYING OUT THE INVENTION

[2125] This invention relates to a system that collects and analyzes personal data and generates a digital personality using a generative artificial intelligence model. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it aims to achieve more human-like responses.

[2126] User Data Collection Phase

[2127] Users upload their photos, videos, voice messages, text data, etc. to a specific application or web portal via their smartphone or PC. The device receives the uploaded data and converts it locally into a standard format (e.g., JSON, XML). The conversion process includes adding metadata and matching formats.

[2128] Data transmission and analysis

[2129] The device sends the converted data to the server. A secure communication protocol such as HTTPS is used for transmission, ensuring data safety. The server receives the data sent from the device and temporarily stores it in storage. At this stage, verification is performed to confirm the data's integrity and consistency.

[2130] The server then analyzes the received data to extract characteristic information such as the user's tone of voice, vocabulary, facial expressions, and behavioral patterns. This analysis uses natural language processing (NLP) techniques and image recognition algorithms. The server then uses an emotion engine to recognize the user's emotional state from the information obtained through data analysis. For example, it can determine whether the user is happy, sad, angry, or surprised.

[2131] Model training and saving

[2132] The server uses a generative artificial intelligence model to learn the user's digital personality based on the extracted characteristic information and emotional information. Deep learning technology is used to train the neural network. The learned digital personality model is stored in storage and is periodically updated with new data, ensuring that the digital personality reflects the latest information.

[2133] Digital personality generation and interface adaptation

[2134] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The generated digital personality faithfully reproduces the user's characteristics and emotions. The server also configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.).

[2135] Interaction Phase

[2136] Users can interact with their digital persona through the device. The device captures input from the user (voice, text, actions, etc.) and sends it to the server in real time. The server then uses the received input data to analyze the user's emotional state using an emotion engine, and generates a response based on the analysis results. The generated response is sent from the server to the device and presented to the user.

[2137] Specific examples

[2138] Specific examples of remote conversations

[2139] 1. A user uses their smartphone to say, "Dad, I had a hard time at work today."

[2140] 2. The device converts the voice into text using automatic speech recognition (ASR) technology and sends it to the server.

[2141] 3. The server analyzes the received text using NLP algorithms and uses an emotion engine to recognize the user's emotions, such as stress or fatigue.

[2142] 4. The server generates a response based on the user's emotional information and past data: "That must have been tough. What happened?"

[2143] 5. The device converts the generated text into speech using text-to-speech (TTS) technology and outputs it through the speaker.

[2144] Specific examples of conversations using AR devices

[2145] 1. A user puts on an AR headset and talks to a virtual digital personality of their mother in their living room.

[2146] 2. The device captures the user's voice and movements using a camera and sensors and sends them to the server.

[2147] 3. The server analyzes the user's comments and actions using NLP and an emotion engine and determines that the user is happy.

[2148] 4. The server generates a response saying, "Wow, that's a nice photo. Where did you take it?"

[2149] 5. The device uses audio and AR images to present the response to the user.

[2150] The system allows users to interact naturally with their digital persona, maintaining an emotional connection through emotionally aware responses. Examples of prompts include "I'm so tired today. Tell me something uplifting," "Let's talk about what happened with your friend yesterday," and "Mom, what do you think about the news these days?"

[2151] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2152] Step 1: Upload data

[2153] Users use their smartphones or PCs to upload personal data such as photos, videos, voice messages, and text data to specific applications or web portals. These data are input, and the device receives the data.

[2154] Step 2: Data conversion

[2155] The terminal converts the uploaded data it receives into a standard format (e.g., JSON, XML). This conversion process includes changing the format of the data and adding metadata. The input raw data is reformatted, and the output is data in a standard format.

[2156] Step 3: Send data

[2157] The device sends the converted data to the server, using a secure communication protocol such as HTTPS to ensure the data is secure. The input is data in a standard format, and the output is sent to the server.

[2158] Step 4: Data reception and storage

[2159] The server receives data sent from the device and temporarily stores it in storage. It verifies the received data to ensure its integrity and consistency. It also verifies the input data and outputs it as stored data.

[2160] Step 5: Data analysis

[2161] The server analyzes the received data and extracts characteristic information such as the user's tone of voice, language, facial expressions, and behavioral patterns. The analysis uses natural language processing (NLP) technology and image recognition algorithms. The input data is analyzed and the extracted characteristic information is output.

[2162] Step 6: Emotion Recognition

[2163] The server uses an emotion engine based on the characteristic information obtained from the data analysis to recognize the user's emotional state, for example, joy, sadness, anger, surprise, etc. The input is the analyzed characteristic information, and the output is the recognized emotion information.

[2164] Step 7: Model training

[2165] The server uses a generative artificial intelligence model to train a digital personality based on the extracted trait and emotion information. This process uses deep learning techniques to train a neural network. The input is the trait and emotion information, and the output is the trained model.

[2166] Step 8: Save the model

[2167] The server saves the trained digital personality model in storage, and periodically updates the model with new data. The saved model is obtained as output.

[2168] Step 9: Digital Persona Generation

[2169] The server generates a digital personality that responds in real time based on the user's data, a trained AI model, and an emotion engine. The input data is the user's real-time information, and the output is a response.

[2170] Step 10: Interface Adaptation

[2171] The server configures the generated digital personality to adapt to multiple interfaces (smartphones, PCs, AR / VR devices, robots, etc.), converts data formats, and generates output appropriate for each interface.

[2172] Step 11: Receiving User Input

[2173] The device captures input from the user (voice, text, movement, etc.). When using an AR device or robot as an action, the input is obtained using a camera or sensor. The captured data becomes the input.

[2174] Step 12: Send Input

[2175] The device sends the captured user input data to the server in real time. In the case of voice data, it is converted to text using ASR technology before being sent. The input is the captured data, and the data sent to the server is the output.

[2176] Step 13: Response Generation

[2177] The server uses the digital personality to generate the optimal response based on the input data and emotional information it receives. Analysis and response generation are performed based on the input data, and the generated response is obtained as output.

[2178] Step 14: Send response

[2179] The server converts the generated response into an appropriate format (audio file, text data, etc.) and sends it to the device. An optimized response is generated as output and sent to the device.

[2180] Step 15: Present the response

[2181] The device presents the response received from the server to the user. Voice responses are output from the speaker, and text responses are displayed on the screen. AR / VR devices use audio and video for output. The presented response is the final output.

[2182] (Application example 2)

[2183] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2184] Conventional customer service systems have difficulty in responding to customer emotions and personal characteristics, making it difficult to achieve human-like dialogue. This has made it difficult to improve customer satisfaction in brick-and-mortar stores. It has also been difficult to provide appropriate product introductions and personalized services.

[2185] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on individuals, means for analyzing the collected data and extracting personal characteristics, means for training a generative artificial intelligence model based on the extracted characteristics and generating a digital personality, means for adapting the generated digital personality to multiple interfaces, means for capturing input from customers and analyzing and responding in real time, means for recognizing emotions and generating appropriate responses, and means for providing customer service. This enables more human-like interactions and service provision that reflects the customer's emotions and personal characteristics.

[2186] "Personal data" refers to information such as an individual's photos, videos, voice messages, and text data.

[2187] "Means of collection" refers to the methods and devices used to obtain data using devices such as smartphones and PCs.

[2188] "Means of analysis" refers to the technology used to analyze the acquired data and extract individual characteristics and behavioral patterns.

[2189] "Personal characteristics" refer to an individual's unique characteristics, such as tone of voice, language, facial expressions, and behavioral patterns.

[2190] A "generative artificial intelligence model" refers to an artificial intelligence that learns based on collected data and has specific functions and capabilities.

[2191] "Digital personas" refer to digital agents created using generative artificial intelligence models that replicate human characteristics and emotions.

[2192] "Means of adaptation" refers to technology that allows the generated digital personality to be compatible with multi...

Claims

1. the means by which data relating to individuals is collected; A means for analyzing the collected data and extracting personal characteristics; A means for training a generative artificial intelligence model based on the extracted characteristics to generate a digital personality; and a means for adapting the generated digital persona to multiple interfaces; A means of capturing user input and analyzing and responding in real time; A system including:

2. 10. The system of claim 1, further comprising means for converting data collected from the user into a standard format.

3. 10. The system of claim 1, further comprising encryption means for ensuring security of collected data.

Images