System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting and pre-processing communication data of a user; means for using a generative model to generate potential search keywords from the pre-processed communication data; and means for integrating the generated search keywords into a suggest function of a search engine.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem with large-scale integrated platforms is that they are unable to provide users with killer content or clear value additions. Furthermore, they lack a mechanism for efficiently extracting users' potential interests and providing appropriate search suggestions based on them. This makes the user's search intent less easily reflected, making the search process more cumbersome. [Means for solving the problem]

[0005] The present invention provides a means for collecting and preprocessing user communication data. It also provides a means for extracting potential search keywords from the preprocessed communication data using a generative model. Finally, it provides a means for integrating the generated search keywords into the search engine's suggestion function, effectively reminding users of the information they want to know and facilitating their search. This system significantly improves the user experience by providing useful search suggestions while ensuring privacy protection.

[0006] "User Communication Data" means information data such as messages, text, images, and voice that a User sends or receives on a communication platform.

[0007] "Preprocessing" refers to a series of processes that filter unnecessary information (e.g., stamps, images, videos, etc.) from raw data and format it into a form suitable for a generative model.

[0008] A "generative model" is an algorithm or system that uses artificial intelligence techniques, particularly natural language processing techniques, to analyze data and generate potential search keywords.

[0009] "Search keywords" are important words or phrases that users enter into search engines to retrieve relevant information.

[0010] A "search engine" is software or a system that searches for information on the Internet based on specific keywords and presents the results.

[0011] The "suggestion function" is a feature that recommends appropriate search keywords and phrases in real time when a user types characters into the search box.

[0012] "Anonymization" refers to a technique or method for processing data so that individuals cannot be identified, thereby protecting privacy.

[0013] "Privacy protection" refers to all measures taken to safely protect users' personal information and privacy-related data and prevent unauthorized access or use by third parties. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a system that collects and analyzes user communication data and integrates it into the suggestion function of a search engine. This system can effectively suggest information that users have forgotten about or that they potentially want to know.

[0036] What the program does

[0037] 1. Collecting chat data

[0038] The server collects chat data from users' devices and stores it in an encrypted database. This collection process is performed periodically, allowing users to view their most recent chat data.

[0039] 2. Data Preprocessing

[0040] The server formats the collected chat data into a format suitable for the generative model. Specifically, it removes unnecessary information (e.g., stamps, images, and videos) from the text data and divides it into sentences. This process allows the generative model to effectively extract keywords.

[0041] 3. Data input to the generative model

[0042] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT and GPT. The model analyzes the chat data and extracts keywords that users are likely to be interested in.

[0043] 4. Keyword extraction and filtering

[0044] A generative model generates important keywords from the chat data, which are then further filtered to remove spam and inappropriate content, resulting in highly reliable keywords.

[0045] 5. Reflection in search suggestions

[0046] The server integrates the extracted keywords into the search engine's suggestion function, specifically by uploading the generated keywords as search suggestion data, which are displayed when a user starts typing in the search box.

[0047] 6. Displaying suggestions to users

[0048] When a user uses a search engine on their device, customized search suggestions are displayed, allowing the user to easily find relevant information.

[0049] Specific examples

[0050] Case 1: Movie talk

[0051] 1. User A and User B have a conversation on LINE about "Do you know what movies are out recently?"

[0052] 2. The server collects and pre-processes this data.

[0053] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[0054] 4. The server integrates these keywords into the search engine's suggestion function.

[0055] 5. When User A uses Yahoo Search, suggestions such as "Recent Movies" and "Movies 2023" are displayed, and the user can click on them to search for more information.

[0056] Case 2: Talking about the weather

[0057] 1. User C sends a message on LINE saying, "Do you know what the weather will be like tomorrow?"

[0058] 2. The server collects and preprocesses this message.

[0059] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[0060] 4. The server integrates these keywords into the search engine's suggestion function.

[0061] 5. When User C uses Yahoo Search, keywords such as "tomorrow's weather" and "weather forecast" are displayed in the suggestion box, allowing them to search directly.

[0062] Privacy and Data Security

[0063] To protect user privacy during data collection and analysis, the server anonymizes the collected data so that it cannot be used to identify individuals, and uses encryption protocols (e.g., TLS / SSL) for data transmission.

[0064] Users can opt out of this suggestion feature from the settings screen, providing peace of mind regarding their privacy.

[0065] In this way, the present invention provides a system that utilizes users' communication data to understand their latent search intent and reflect this in the search engine's suggestion function, thereby facilitating users' search activities.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] The server collects chat data from user devices, retrieves it in real time or periodically via API, and stores it in a secure database. The data is protected using encryption technology while stored.

[0069] Step 2:

[0070] The server preprocesses the collected talk data into a format suitable for the generative model. Specifically, unnecessary information such as stamps, images, and videos are filtered from the text data, and the data is divided into talks and formatted as text tokens.

[0071] Step 3:

[0072] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT or GPT. This model analyzes the chat data and extracts search keywords that the user is likely to be interested in.

[0073] Step 4:

[0074] The generative model extracts keywords from the analysis of the chat data. Specifically, it selects and lists highly important keywords and phrases. These keywords then undergo a filtering process to remove inappropriate content and spam.

[0075] Step 5:

[0076] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, it uploads the generated keywords as suggestion data through the Yahoo! Search API. This upload process immediately updates the keywords to the suggestion list.

[0077] Step 6:

[0078] When a user uses a search engine on their device, the search engine displays customized search suggestions. As the user types in the search box, the keywords integrated in step 5 are displayed in real time.

[0079] Step 7:

[0080] Users can click on the suggested keywords and perform a search to retrieve related information, allowing them to quickly access the information they were potentially looking for.

[0081] Step 8:

[0082] The server collects data on users' search behavior and uses it to improve the system. Specifically, it collects data such as which suggestions users clicked and which search results they viewed, and reuses this data as training data for the generative model.

[0083] Step 9:

[0084] The generative model is retrained based on the collected feedback data, which improves the accuracy of keyword extraction the next time and allows the model to provide more appropriate suggestions to users.

[0085] Step 10:

[0086] Users can opt out of data collection and suggestion features in the settings screen, allowing them to use the service in a privacy-conscious manner.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] The suggestion functions of conventional search engines are based on users' past search history and general trends, and therefore often fail to effectively provide keywords that are relevant to the user's current interests and circumstances. Furthermore, from the perspective of privacy protection, there are sometimes concerns about whether user data is being managed appropriately. Therefore, there is a need for the development of a system that reflects the information users are seeking in everyday conversations in real time while protecting their privacy.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for collecting and preprocessing user communication data, means for removing unnecessary information from the preprocessed communication data and dividing it into sentence units, means for inputting the divided communication data into a generative model to generate potential search keywords, and means for filtering the generated search keywords and integrating them into the search engine's suggestion function. This makes it possible to provide a suggestion function that reflects the user's latest interests and situation, thereby achieving both an improved user experience and privacy protection.

[0092] "Communication data" refers to text information exchanged by users using chat applications, messaging services, etc.

[0093] "Preprocessing" is the process of removing unnecessary information from collected communication data and formatting it into a format that is easy to analyze.

[0094] A "generative model" is an algorithm or software that uses natural language processing techniques to generate potential search keywords from preprocessed communication data.

[0095] A "search engine suggestion function" is a search assistance function that automatically suggests related keywords when a user begins typing in the search box.

[0096] "Filtering" is the process of removing inappropriate content and spam from the generated keywords, leaving only reliable keywords.

[0097] An "encrypted database" is a database that is protected using encryption technology to ensure data security.

[0098] "Anonymization" is a process that protects privacy by removing personally identifiable information from collected communications data.

[0099] The present invention is a system that collects and preprocesses user communication data, and then integrates keywords generated from that data into the suggestion function of a search engine, thereby providing suggestion functions that reflect the user's latest interests and circumstances.

[0100] Hardware and software used

[0101] This system uses the following hardware and software:

[0102] Server: A central computing unit for data collection, preprocessing, running generative models, filtering keywords, and integrating them into suggestion functions. Specific examples include high-performance cloud servers (e.g., AWS, GCP).

[0103] User terminal: A device that generates communication data and displays suggestions. Examples include smartphones, tablets, and PCs.

[0104] Generative AI model: A generative model that uses natural language processing techniques. In particular, we use the latest generative AI models such as BERT and GPT.

[0105] Data processing and calculation

[0106] 1. Collecting Talk Data:

[0107] The server periodically collects chat data from the user's device. This process involves obtaining message data from chat apps such as the LINE application.

[0108] The collected data is stored in an encrypted database to ensure security.

[0109] 2. Data preprocessing:

[0110] The server retrieves the collected data from the encrypted database and removes unnecessary information (e.g. stamps, images, videos).

[0111] The preprocessed data is split into text sentences and converted into a clean format, removing special characters and unnecessary spaces in the process.

[0112] 3. Data input to the generative model:

[0113] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4), which generates potential search keywords from the user's conversation.

[0114] The generated keywords are output along with a confidence score.

[0115] 4. Keyword extraction and filtering:

[0116] The server runs the keywords obtained from the generative AI model through a filtering algorithm to remove inappropriate content and spam.

[0117] The filtered keywords are saved as a reliable list.

[0118] 5. Reflection in search suggestions:

[0119] The server uploads the filtered keywords to the search engine's suggestion database, which is updated in real time.

[0120] 6. Suggestions for users:

[0121] The user's device will display customized search suggestions through the browser or search application, allowing the user to efficiently access the information they need.

[0122] Specific examples

[0123] Some specific examples are given below.

[0124] Case 1: Movie talk

[0125] 1. User A and User B have a conversation about "Do you know what movies are out recently?"

[0126] 2. The server collects and preprocesses this talk data.

[0127] 3. The generative model generates keywords such as "recent movies" and "movies 2023."

[0128] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[0129] 5. When User A performs a web search, suggestions such as "recent movies" and "movies 2023" are displayed.

[0130] Case 2: Talking about the weather

[0131] 1. User C sends a message saying, "Do you know what the weather will be like tomorrow?"

[0132] 2. The server collects and preprocesses this message.

[0133] 3. The generative model generates keywords such as "tomorrow's weather" and "weather forecast."

[0134] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[0135] 5. When User C uses a search engine, keywords such as "tomorrow's weather" and "weather forecast" are suggested.

[0136] This mechanism makes it possible to provide effective suggestions based on the user's latest interests.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1:

[0139] The server collects chat data from the user's device. Specifically, it obtains the message history of the chat application. For example, it collects message data from LINE such as "Do you know what movies are out these days?" The input is the user's communication data, and the output is the result of collecting this data. The data is encrypted and stored in a secure database.

[0140] Step 2:

[0141] The server preprocesses the collected chat data. The input is chat data read from an encrypted database, and the output is clean data with unnecessary information removed. Specifically, non-text information such as stamps, images, and videos in messages is removed, and the text is divided into sentences. Special characters and unnecessary spaces are also removed during this process.

[0142] Step 3:

[0143] The server inputs the preprocessed data into a generative model. The input is cleaned and formatted talk data, and the output is generated keywords and their confidence scores. Specifically, a generative AI model (e.g., GPT-4) is used to analyze the text and generate potential search keywords such as "recent movies" and "movies 2023."

[0144] Step 4:

[0145] The server inputs the keywords obtained from the generative model into a filtering algorithm. The input is the generated keywords, and the output is a filtered, more reliable list of keywords. Specific filtering actions include removing spam and inappropriate content. For example, if the keyword "recent movies" is appropriate, it will be left as is, but "spammy content" will be removed.

[0146] Step 5:

[0147] The server uploads the filtered keyword list to the search engine's suggestion database. The input is the filtered keyword list, and the output is a search suggestion function that integrates the list. Specific operations include writing to the database and updating it in real time. This process adds keywords such as "movies 2023" to the suggestion database.

[0148] Step 6:

[0149] When a user uses a search engine on their device, the generated keywords are displayed as suggestions in real time. The input is the user's search query, and the output is the keywords displayed in the suggestion box. Specifically, when a user begins to type in the search box, keywords such as "recent movies" and "weather forecast" are presented as candidates. As a result, users can efficiently access related information.

[0150] (Application example 1)

[0151] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0152] Conventional navigation systems are unable to understand the individual needs and preferences of users, and are limited to providing pre-set route information and destinations. This means that they are unable to appropriately suggest places and information that are truly of interest to users. In relation to this, there is a need for real-time, personalized navigation based on in-car conversations and behavior.

[0153] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0154] In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative AI model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into a navigation system, and means for displaying the keywords generated based on conversation data collected in the car on the navigation system. This enables real-time keyword generation based on conversation data in the car and personalized destination and information suggestions based on the generated keywords.

[0155] "User communications data" is information generated by users, such as conversations in the car or text messages.

[0156] "Preprocessing" refers to the process of removing noise and unnecessary information from collected user communication data and shaping the data to make it easier to analyze using a generative model.

[0157] A "generative model" is a model that extracts latent keywords from user communication data that has been preprocessed using natural language processing technology. Specifically, AI models such as BERT and GPT fall into this category.

[0158] "Integrating into the navigation system" refers to the process of incorporating the keywords extracted by the generative model into the database of the vehicle's navigation system, enabling information to be provided in real time.

[0159] A "navigation system" is an electronic system installed in a vehicle that allows the user to set a destination and provides optimal route guidance.

[0160] "Conversational data collected in the vehicle" refers to the user's voice and text information obtained through the vehicle's microphone and text input device.

[0161] "Keywords" are important words or phrases extracted from communication data by the generative model and used in navigation systems to identify specific information or destinations.

[0162] "Displaying" refers to visually providing information and destinations related to the extracted keywords to the user on the display of the navigation system.

[0163] A system for implementing this invention is a navigation system installed in an autonomous vehicle that generates potential keywords based on user communication data and uses them to provide personalized navigation guidance.

[0164] System Configuration

[0165] The system includes the following major components:

[0166] 1. In-vehicle server

[0167] 2. Generative Model

[0168] 3. Navigation system

[0169] 4. In-car displays

[0170] 5. Microphones and Text Input Devices

[0171] What the program does

[0172] In-vehicle server

[0173] The in-vehicle server collects conversation data through the vehicle's microphone and text input devices. The collected data is stored in an encrypted database. This collection process is performed periodically to obtain the latest data.

[0174] Pretreatment

[0175] The in-vehicle server preprocesses the collected conversation data into a format that is easy to analyze. Specifically, it removes noise and unnecessary information and divides the data into sentences. This preprocessing allows the generative model to effectively extract keywords.

[0176] Generative Model

[0177] The preprocessed conversation data is input to a generative model (e.g., BERT or GPT), which analyzes the data and extracts keywords that the user is likely to be interested in from the conversations in the car.

[0178] Integration into navigation systems

[0179] The in-vehicle server integrates the generated keywords into the navigation system, which then uses these customized keywords to display personalized routes and information when the user sets a destination.

[0180] Displaying suggestions to users

[0181] The in-car display will show customized suggestions through the navigation system, allowing users to easily find relevant information and enjoy greater convenience.

[0182] Hardware and software used

[0183] The system includes the following specific hardware and software:

[0184] In-vehicle server: A server installed in the vehicle for collecting and analyzing data

[0185] Microphone and text input device: A device for collecting voice data and text information inside the vehicle.

[0186] Generative models: AI models that use natural language processing techniques such as BERT and GPT

[0187] Navigation system: In-vehicle route guidance and information display system

[0188] Encryption protocol: Technology to ensure security of data communication such as TLS / SSL

[0189] Examples and prompts

[0190] Specific examples

[0191] Case 1: A user asks in the car, "Is there a good ramen restaurant nearby?"

[0192] An in-vehicle server collects and pre-processes this data.

[0193] GPT-4 extracts keywords such as "delicious ramen restaurant" and "recommended ramen restaurant."

[0194] The extracted keywords are integrated into the navigation system.

[0195] When the user launches the navigation system, suggestions such as "delicious ramen restaurants" are displayed, and navigation can be started immediately.

[0196] Prompt Sentence Examples

[0197] Generate keywords that will attract users' interest based on the latest in-car conversation data. Example keywords are shown below.

[0198] "Delicious ramen restaurant"

[0199] "Nearby gas station"

[0200] "Playground for children"

[0201] In this way, the system of the present invention can improve the convenience and personalized experience of autonomous vehicles by utilizing the user's real-time communication data and enhancing the navigation system's suggestion functions.

[0202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0203] Step 1:

[0204] The server collects the user's communication data (conversation data and text messages) from the microphone and text input device. The input is the user's real-time conversation data, and the output is formatted data stored in an encrypted database. This collection process is performed periodically to capture the latest communication data occurring inside the vehicle.

[0205] Step 2:

[0206] The server preprocesses the collected conversation data. Specifically, it removes noise and unnecessary information and divides it into sentences. The input is raw data from an encrypted database, and the output is noise-removed, formatted text data. This preprocessing allows the generative model to effectively extract keywords.

[0207] Step 3:

[0208] The server inputs the preprocessed conversation data into a generative model (e.g., GPT-4 or BERT). The generative model analyzes the conversation data and extracts keywords of potential interest. The input is preprocessed text data, and the output is a list of keywords as the analysis result.

[0209] Step 4:

[0210] The server integrates the generated keywords into the navigation system. Specifically, it adds the generated keywords to the navigation system database or displays them as real-time search suggestions. The input is the generated keyword list, and the output is the suggestion data integrated into the navigation system.

[0211] Step 5:

[0212] The navigation system will suggest destinations and related information to the user based on the generated keywords through the in-car display. The input is the suggestion data integrated into the navigation system, and the output is customized destination and route information displayed on the display. This step allows the user to intuitively browse customized destination information.

[0213] Step 6:

[0214] When the user selects a suggested keyword on the in-car display, the navigation system generates the optimal route based on the selected destination and information and begins providing guidance. The input is the user's selected keyword, and the output is the specific guided route.

[0215] This series of processing steps allows users to enjoy personalized navigation based on the conversations they have in the car, greatly improving convenience.

[0216] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0217] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines an emotion engine that recognizes the user's emotions to provide more appropriate search suggestions based on the user's emotional state. This makes it possible to further personalize the user experience and improve satisfaction.

[0218] What the program does

[0219] 1. Collecting chat data

[0220] The server collects chat data from user devices and stores it in encrypted form in a secure database. This collection can occur in real time or periodically.

[0221] 2. Data Preprocessing

[0222] The server preprocesses the collected chat data, specifically filtering out unnecessary information (e.g., stickers, images, videos) and formatting it into a format suitable for the generative model.

[0223] 3. Data input to the generative model

[0224] The server inputs the preprocessed chat data into a generative model (e.g., BERT or GPT), which analyzes this data and extracts keywords that potentially interest the user.

[0225] 4. Introducing the Emotion Engine

[0226] The server inputs the same data into the emotion engine in parallel with the conversation data. The emotion engine uses natural language processing technology to recognize emotions from the user's conversation.

[0227] 5. Keyword extraction and sentiment consideration

[0228] The generative AI model analyzes the chat data to generate important keywords, and the emotion engine weights the keywords based on the user's emotional data. For example, if the user is expressing positive emotions, it will prioritize keywords that match that mood.

[0229] 6. Keyword Filtering

[0230] The server filters the generated keywords to remove inappropriate content and spam, a process that also takes sentiment data into account.

[0231] 7. Reflection in suggestion function

[0232] The server then integrates the extracted keywords into the search engine's suggestion function, taking into account emotional data. Specifically, it uses the Yahoo! Search API to upload appropriate keywords as suggestion data.

[0233] 8. Displaying suggestions to users

[0234] When a user uses a search engine on their device, they will see personalized search suggestions based on their emotions, helping them quickly find the information that best matches their search intent.

[0235] Specific examples

[0236] Case 1: Movie talk

[0237] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[0238] 2. The server collects and preprocesses this talk data.

[0239] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[0240] 4. The emotion engine recognizes user A's positive emotions and weights positive movie keywords that match those emotions.

[0241] 5. The server integrates these keywords into the search engine's suggestion function, and when User A uses Yahoo Search, emotion-based suggestions such as "recent movies" and "movies 2023" are displayed.

[0242] Case 2: Talking about the weather

[0243] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[0244] 2. The server collects and preprocesses this message.

[0245] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[0246] 4. The emotion engine recognizes user C's anxious emotion and weights keywords for detailed weather information that correspond to that emotion.

[0247] 5. The server integrates these keywords into the search engine's suggestion function, and when User C uses Yahoo Search, detailed suggestions such as "tomorrow's weather" and "weather forecast" based on emotions are displayed.

[0248] Privacy and Data Security

[0249] During the data collection and analysis process, the server anonymizes the collected data to protect user privacy and makes it impossible to identify individuals. In addition, encryption protocols (e.g., TLS / SSL) are used for data transmission.

[0250] Users can opt out of data collection, sentiment analysis, and suggestion functions from the settings screen, allowing them to use the service in a privacy-conscious manner.

[0251] The present invention significantly improves user experience by providing search suggestions that take into account the user's communication data and their sentiments.

[0252] The processing flow will be explained below.

[0253] Step 1:

[0254] The server collects chat data from user devices, periodically retrieves the data via API, and stores it in a secure database. The data is protected using AES encryption technology when stored.

[0255] Step 2:

[0256] The server preprocesses the collected chat data. Specifically, it filters unnecessary information (e.g., stamps, images, videos) from the text data, divides it into chats, and formats it appropriately.

[0257] Step 3:

[0258] The server inputs the preprocessed talk data into a generative model (e.g., a natural language processing model such as BERT or GPT), which analyzes the talk data and extracts important keywords.

[0259] Step 4:

[0260] The server also inputs the chat data into the emotion engine, which uses natural language processing technology to analyze and recognize emotions from the user's text. The analysis results are output as emotion labels such as anger, joy, and sadness.

[0261] Step 5:

[0262] The generative model generates important keywords based on the preprocessed chat data. At this time, the emotion engine references the emotional data recognized and weights the keywords. For example, if the user is expressing positive emotions, it will prioritize keywords related to happy content.

[0263] Step 6:

[0264] The server-generated keywords are filtered to remove inappropriate content and spam from the keyword list and select only reliable keywords. Sentiment data is also taken into account in this filtering process.

[0265] Step 7:

[0266] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, the keywords are uploaded as suggestion data via the search engine's API. This allows emotion-sensitive keywords to be displayed as suggestions in the search box.

[0267] Step 8:

[0268] When a user uses a search engine on their device, search suggestions customized based on their emotions are displayed. When a user types text into the search box, keywords integrated by the server are displayed as suggestions in real time.

[0269] Step 9:

[0270] Users can click on suggested keywords and perform a search to quickly retrieve relevant information, which is a process that best matches the user's search intent and increases user satisfaction.

[0271] Step 10:

[0272] The server collects data on users' search behavior and uses it to improve the system, such as which suggestions were clicked and which search results were viewed, and uses this data to train the generative model and emotion engine.

[0273] Step 11:

[0274] The generative model is retrained based on the collected feedback data to improve its keyword extraction accuracy the next time, which allows it to provide more relevant suggestions to users.

[0275] Step 12:

[0276] Users can opt out of data collection, sentiment analysis, and suggestion functions through the settings screen. This protects users' privacy and provides an environment where users can use the service with peace of mind.

[0277] Example 2

[0278] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0279] The suggestion function of conventional search engines presented keywords without taking into account the user's emotions or the content of their communication data, which meant that they were unable to fully meet the needs of users. In addition, they lacked privacy protection, making it difficult to provide an environment where users could use the search engine with peace of mind.

[0280] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for weighting the generated search keywords based on the user's emotion data, means for filtering the generated keywords to remove inappropriate content and spam, and means for integrating the generated search keywords into the suggestion function of the search engine. This makes it possible to provide personalized search suggestions that reflect the user's emotions and the content of the communication data, while also ensuring privacy protection.

[0281] "Communications Data" refers to information such as text messages, audio data, and video data that a user sends or receives.

[0282] "Preprocessing" is the process of removing unnecessary information from collected communication data and converting it into a format suitable for analysis and processing.

[0283] A "generative model" is an algorithm or machine learning model that uses natural language processing technology to extract potential search keywords from preprocessed communication data.

[0284] "Emotion data" is data that indicates a user's emotions and psychological state, recognized from communication data using natural language processing technology.

[0285] "Weighting" is the process of assigning a certain score or priority to the generated keywords to adjust their importance.

[0286] "Filtering" is the process of removing inappropriate content and spam from the generated keywords.

[0287] A "suggestion function" is a function that automatically suggests related keywords in response to a query entered by a user in a search engine.

[0288] "Privacy protection" refers to the means and technologies that anonymize users' personal information and communication data and protect them from leaks to third parties.

[0289] "Encryption" is the technology of transforming data using specific algorithms so that only authorized persons can decipher it.

[0290] "Data collection" is the process of importing user communication data via a network into a server or database.

[0291] "Natural language processing" is the field of technology that enables computers to understand, generate, and manipulate human language.

[0292] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines this with an emotion engine that recognizes the user's emotions, to provide more appropriate search suggestions based on the user's emotional state.

[0293] The server first collects user communication data. This collected data is then stored in a secure database in encrypted form using AES 256-bit encryption. The collected data is then sent to the server in real time or periodically.

[0294] The server then pre-processes the collected communication data. This involves filtering out media data such as stamps, images, and videos from the text. This data is then tokenized using a morphological analysis tool such as MeCab, and converted into a format suitable for analysis and processing.

[0295] The preprocessed data is then input into a generative model, such as BERT or GPT, which is implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model understands the context of the chat data and extracts keywords that potentially interest users.

[0296] In parallel, the server also inputs the same chat data into the emotion engine, which uses natural language processing (NLP) to recognize the user's emotions from the communication data. Emotions are classified into labels such as positive, negative, and neutral. Tools such as TextBlob and VADER are used for this emotion analysis.

[0297] The keywords extracted by the generative AI model are then weighted based on the emotional data recognized by the emotion engine. The weighting is done using a scoring algorithm to prioritize keywords that match a user's positive emotion.

[0298] The generated keywords then go through a filtering process, which removes inappropriate content and spam. This includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to remove inappropriate keywords, improving the quality of the user experience.

[0299] The filtered keywords are finally integrated into the search engine's suggestion function. Appropriate keywords are uploaded as suggestion data using the Yahoo! Search API, etc. This updates the suggestion database in real time.

[0300] When users use search engines, they will see personalized search suggestions, which will give users search suggestions optimized based on sentiment. For example, if you search for "recent movies," you will see relevant suggestions such as "movies 2023" and "trending movies."

[0301] Specific examples include the following cases:

[0302] Case 1: Movie talk

[0303] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[0304] 2. The server collects this chat data, encrypts it, and stores it in a database.

[0305] 3. The server preprocesses and tokenizes the text data.

[0306] 4. The generative model extracts keywords such as "recent movies" and "movies 2023" from the preprocessed data.

[0307] 5. The emotion engine recognizes User A's positive emotions and weights positive movie keywords.

[0308] 6. The server filters these keywords and removes inappropriate content.

[0309] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[0310] 8. When User A uses a search engine, suggestions such as "recent movies" and "movies 2023" appear.

[0311] Case 2: Talking about the weather

[0312] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[0313] 2. The server collects this message, encrypts it, and stores it in a database.

[0314] 3. The server preprocesses and tokenizes the text data.

[0315] 4. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast" from the preprocessed data.

[0316] 5. The emotion engine recognizes user C's anxious emotions and weights keywords for detailed weather information.

[0317] 6. The server filters these keywords and removes inappropriate content.

[0318] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[0319] 8. When User C uses a search engine, they will see emotion-based suggestions such as "tomorrow's weather" and "weather forecast."

[0320] An example of a prompt is:

[0321] "Are there any good movies out recently?"

[0322] "What's the weather going to be like tomorrow?"

[0323] In this way, the present invention can efficiently provide personalized search suggestions that take into account the user's communication data and emotions.

[0324] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0325] Step 1:

[0326] The server collects communication data from the user's device. Specifically, when a user chats using a messaging app, the text data is sent to the server via the network. At this time, the collected communication data is stored in a database using AES 256-bit encryption. The input is a text message from the user, and the output is encrypted communication data.

[0327] Step 2:

[0328] The server preprocesses the collected communication data. Specifically, it filters non-text information such as stamps, images, and videos from the text data, and divides the text into words using tokenization and morphological analysis tools (e.g., MeCab). The input to this step is the encrypted and stored communication data, and the output is clean text data after preprocessing.

[0329] Step 3:

[0330] The server inputs the preprocessed text data into a generative model. Specifically, it applies generative models such as BERT or GPT, which are implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model analyzes and extracts keywords that potentially interest the user from the preprocessed data. The input is the preprocessed text data, and the output is the extracted latent keywords.

[0331] Step 4:

[0332] The server simultaneously inputs the preprocessed text data into the emotion engine. The emotion engine uses natural language processing technology to recognize user emotions from the communication data. Specifically, it uses tools such as TextBlob and VADER to assign emotion labels such as positive, negative, and neutral to the data. The input is the preprocessed text data, and the output is the assigned emotion label.

[0333] Step 5:

[0334] The generative AI model weights the extracted keywords based on emotional data obtained from the emotion engine. For example, if a user expresses a positive emotion, the scoring algorithm is used to prioritize keywords that match that mood. The inputs are the extracted keywords and emotion labels, and the output is the weighted keywords.

[0335] Step 6:

[0336] The server filters the generated keywords to remove inappropriate content and spam. This process includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to ensure a good user experience. The input is weighted keywords, and the output is clean, filtered keywords.

[0337] Step 7:

[0338] The server finally integrates the filtered keywords into the search engine's suggestion function. Specifically, by uploading appropriate keywords as suggestion data using the search engine API, the suggestion function database is updated in real time. The input is the filtered keywords, and the output is the updated search engine's suggestion data.

[0339] Step 8:

[0340] When using a search engine, a user's device displays customized search suggestions, allowing the user to obtain search results optimized based on sentiment. For example, when searching for "recent movies," relevant suggestions such as "movies 2023" and "trending movies" are displayed. The input is a search query, and the output is search suggestions.

[0341] This allows users to experience personalized search suggestions based on emotions.

[0342] (Application example 2)

[0343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] Conventional search engine suggestion functions are limited to suggesting keywords based on the user's communication data and do not take into account the user's emotional state, making it impossible to provide a truly personalized experience. In addition, user privacy may not be adequately protected, and ensuring safety is also an issue.

[0345] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into the search engine's suggestion function, means for recognizing the user's emotional state from the user's communication data, and means for weighting the generated search keywords based on the recognized emotional state. This makes it possible to suggest search keywords based on the user's emotional state, providing a more personalized user experience. Privacy protection is also ensured by anonymizing the communication data and emotional data.

[0346] "User Communication Data" means all data sent or received by a User through a Communication Device, including, in particular, text messages, voice data, and other information.

[0347] "Preprocessing" refers to a series of processes that remove unnecessary information from collected communication data and convert it into a format suitable for analysis.

[0348] A "generative model" refers to a machine learning algorithm that extracts and generates keywords that may be of interest to users based on collected and preprocessed data.

[0349] "Search engine suggestion function" refers to the function that predicts and displays related search keywords when a user types something into a search box.

[0350] "Means for recognizing emotional states" refers to algorithms that analyze a user's communication data and identify their emotional state (such as joy, sadness, or anxiety).

[0351] "Weighting" refers to the process of prioritizing generated keywords based on emotional state.

[0352] "Anonymization" refers to the process of removing personally identifiable information from collected data to protect user privacy.

[0353] The present invention relates to a system for generating search keywords based on communication data and emotional state of a user in a virtual store, and for making more personalized product suggestions. The system includes the following means.

[0354] A means of collecting and preprocessing user communication data

[0355] The server collects user communication data in real time or periodically via smartphones or head-mounted displays. The collected communication data is stored in an encrypted database (e.g., MySQL) on the server. Data security is ensured using encryption protocols such as TLS / SSL. Preprocessing involves filtering out unnecessary information (e.g., stamps, images, and videos) and converting text data into an appropriate format. This process makes the data easier to analyze.

[0356] A method using a generative model to generate potential search keywords from preprocessed communication data

[0357] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to extract search keywords that the user is likely to be interested in. This generative model can generate appropriate keywords based on past data and user behavior patterns.

[0358] A means to integrate generated search keywords into search engine suggestion functions

[0359] The extracted search keywords are filtered by the server to remove inappropriate content and spam, and then uploaded to the suggestion system (e.g., search engine API) within the virtual store and displayed when a user searches.

[0360] A means of recognizing a user's emotional state from their communication data

[0361] The server inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The emotional state is classified into major emotion categories such as joy, sadness, and anxiety. The recognition results play an important role in the subsequent keyword generation process.

[0362] A means of weighting generated search keywords based on perceived emotional states

[0363] The server adjusts the importance of generated search keywords based on the emotional data obtained from the emotion engine. For example, if a user feels like relaxing, keywords such as "relaxation goods" and "aroma" will be prioritized.

[0364] Specific examples

[0365] Specific scenarios

[0366] 1. User A searches for "relaxing products" in a virtual store.

[0367] 2. A smartphone app collects the user's relaxed emotional state along with their conversation data.

[0368] 3. The server preprocesses the data and inputs it into GPT-3 and the Microsoft Azure Emotion API.

[0369] 4. GPT-3 extracts keywords such as "relaxation," "relaxation goods," and "aroma."

[0370] 5. The Microsoft Azure Emotion API recognizes relaxed emotions.

[0371] 6. The suggestion system weights "relaxation," "relaxation goods," and "aroma" and integrates them into the virtual store's suggestion list.

[0372] 7. When User A browses the virtual store, products such as "relaxation goods" and "aroma" are displayed preferentially.

[0373] Prompt Sentence Examples

[0374] Collect chat data that shows users feeling relaxed and analyze it using an emotion engine. The generative AI model should extract keywords that users are likely to be interested in, such as "relaxation," "aroma," and "relaxation goods," and integrate these into the suggestion system.

[0375] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0376] Step 1:

[0377] The server collects the user's communication data from the smartphone or head-mounted display. The collected communication data is encrypted and stored in a database. The input is the user's communication data (text messages, voice data, and other information), and the output is the data stored in the database in encrypted form. This step includes specific operations to ensure the security of the data using the TLS / SSL protocol.

[0378] Step 2:

[0379] The server preprocesses the collected communication data. Specifically, it filters out unnecessary information (e.g., stamps, images, videos) and formats it as text data. The input is encrypted communication data, and the output is preprocessed text data. In this step, the data cleansing and filtering process is performed.

[0380] Step 3:

[0381] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to generate search keywords that the user is likely to be interested in. The input is the preprocessed text data, and the output is the generated search keywords. This step includes the specific operation of extracting keywords using a machine learning algorithm.

[0382] Step 4:

[0383] The server simultaneously inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The input is preprocessed text data, and the output is the recognized emotional state. In this step, natural language processing techniques are used to classify the user's emotion.

[0384] Step 5:

[0385] The server combines the search keywords obtained from the generative AI model with the emotional data obtained from the emotion engine and weights the keywords. The input is the generated search keywords and the recognized emotional state, and the output is the weighted search keywords. In this step, an algorithm is run to determine the priority of the keywords.

[0386] Step 6:

[0387] The server filters the weighted search keywords to remove inappropriate content and spam. The input is the weighted search keywords, and the output is the filtered keywords. This step includes specific operations to remove inappropriate keywords using text analysis techniques.

[0388] Step 7:

[0389] The server integrates the filtered keywords into the search engine's suggestion function. The input is the filtered keywords, and the output is the keywords uploaded as search engine suggestion data. In this step, the search engine API is used to add the keywords to the suggestion system.

[0390] Step 8:

[0391] When a user uses a virtual store, search suggestions customized based on their emotions are displayed. The input is the search engine's suggestion data, and the output is the customized suggestions displayed on the user's device. This step includes the specific operation of displaying appropriate suggestions through the user interface.

[0392] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0393] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0394] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0395] [Second embodiment]

[0396] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0397] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0398] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0399] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0400] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0401] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0402] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0403] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0404] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0405] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0406] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0407] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0408] This invention is a system that collects and analyzes user communication data and integrates it into the suggestion function of a search engine. This system can effectively suggest information that users have forgotten about or that they potentially want to know.

[0409] What the program does

[0410] 1. Collecting chat data

[0411] The server collects chat data from users' devices and stores it in an encrypted database. This collection process is performed periodically, allowing users to view their most recent chat data.

[0412] 2. Data Preprocessing

[0413] The server formats the collected chat data into a format suitable for the generative model. Specifically, it removes unnecessary information (e.g., stamps, images, and videos) from the text data and divides it into sentences. This process allows the generative model to effectively extract keywords.

[0414] 3. Data input to the generative model

[0415] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT and GPT. The model analyzes the chat data and extracts keywords that users are likely to be interested in.

[0416] 4. Keyword extraction and filtering

[0417] A generative model generates important keywords from the chat data, which are then further filtered to remove spam and inappropriate content, resulting in highly reliable keywords.

[0418] 5. Reflection in search suggestions

[0419] The server integrates the extracted keywords into the search engine's suggestion function, specifically by uploading the generated keywords as search suggestion data, which are displayed when a user starts typing in the search box.

[0420] 6. Displaying suggestions to users

[0421] When a user uses a search engine on their device, customized search suggestions are displayed, allowing the user to easily find relevant information.

[0422] Specific examples

[0423] Case 1: Movie talk

[0424] 1. User A and User B have a conversation on LINE about "Do you know what movies are out recently?"

[0425] 2. The server collects and pre-processes this data.

[0426] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[0427] 4. The server integrates these keywords into the search engine's suggestion function.

[0428] 5. When User A uses Yahoo Search, suggestions such as "Recent Movies" and "Movies 2023" are displayed, and the user can click on them to search for more information.

[0429] Case 2: Talking about the weather

[0430] 1. User C sends a message on LINE saying, "Do you know what the weather will be like tomorrow?"

[0431] 2. The server collects and preprocesses this message.

[0432] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[0433] 4. The server integrates these keywords into the search engine's suggestion function.

[0434] 5. When User C uses Yahoo Search, keywords such as "tomorrow's weather" and "weather forecast" are displayed in the suggestion box, allowing them to search directly.

[0435] Privacy and Data Security

[0436] To protect user privacy during data collection and analysis, the server anonymizes the collected data so that it cannot be used to identify individuals, and uses encryption protocols (e.g., TLS / SSL) for data transmission.

[0437] Users can opt out of this suggestion feature from the settings screen, providing peace of mind regarding their privacy.

[0438] In this way, the present invention provides a system that utilizes users' communication data to understand their latent search intent and reflect this in the search engine's suggestion function, thereby facilitating users' search activities.

[0439] The processing flow will be explained below.

[0440] Step 1:

[0441] The server collects chat data from user devices, retrieves it in real time or periodically via API, and stores it in a secure database. The data is protected using encryption technology while stored.

[0442] Step 2:

[0443] The server preprocesses the collected talk data into a format suitable for the generative model. Specifically, unnecessary information such as stamps, images, and videos are filtered from the text data, and the data is divided into talks and formatted as text tokens.

[0444] Step 3:

[0445] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT or GPT. This model analyzes the chat data and extracts search keywords that the user is likely to be interested in.

[0446] Step 4:

[0447] The generative model extracts keywords from the analysis of the chat data. Specifically, it selects and lists highly important keywords and phrases. These keywords then undergo a filtering process to remove inappropriate content and spam.

[0448] Step 5:

[0449] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, it uploads the generated keywords as suggestion data through the Yahoo! Search API. This upload process immediately updates the keywords to the suggestion list.

[0450] Step 6:

[0451] When a user uses a search engine on their device, the search engine displays customized search suggestions. As the user types in the search box, the keywords integrated in step 5 are displayed in real time.

[0452] Step 7:

[0453] Users can click on the suggested keywords and perform a search to retrieve related information, allowing them to quickly access the information they were potentially looking for.

[0454] Step 8:

[0455] The server collects data on users' search behavior and uses it to improve the system. Specifically, it collects data such as which suggestions users clicked and which search results they viewed, and reuses this data as training data for the generative model.

[0456] Step 9:

[0457] The generative model is retrained based on the collected feedback data, which improves the accuracy of keyword extraction the next time and allows the model to provide more appropriate suggestions to users.

[0458] Step 10:

[0459] Users can opt out of data collection and suggestion features in the settings screen, allowing them to use the service in a privacy-conscious manner.

[0460] Example 1

[0461] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0462] The suggestion functions of conventional search engines are based on users' past search history and general trends, and therefore often fail to effectively provide keywords that are relevant to the user's current interests and circumstances. Furthermore, from the perspective of privacy protection, there are sometimes concerns about whether user data is being managed appropriately. Therefore, there is a need for the development of a system that reflects the information users are seeking in everyday conversations in real time while protecting their privacy.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0464] In this invention, the server includes means for collecting and preprocessing user communication data, means for removing unnecessary information from the preprocessed communication data and dividing it into sentence units, means for inputting the divided communication data into a generative model to generate potential search keywords, and means for filtering the generated search keywords and integrating them into the search engine's suggestion function. This makes it possible to provide a suggestion function that reflects the user's latest interests and situation, thereby achieving both an improved user experience and privacy protection.

[0465] "Communication data" refers to text information exchanged by users using chat applications, messaging services, etc.

[0466] "Preprocessing" is the process of removing unnecessary information from collected communication data and formatting it into a format that is easy to analyze.

[0467] A "generative model" is an algorithm or software that uses natural language processing techniques to generate potential search keywords from preprocessed communication data.

[0468] A "search engine suggestion function" is a search assistance function that automatically suggests related keywords when a user begins typing in the search box.

[0469] "Filtering" is the process of removing inappropriate content and spam from the generated keywords, leaving only reliable keywords.

[0470] An "encrypted database" is a database that is protected using encryption technology to ensure data security.

[0471] "Anonymization" is a process that protects privacy by removing personally identifiable information from collected communications data.

[0472] The present invention is a system that collects and preprocesses user communication data, and then integrates keywords generated from that data into the suggestion function of a search engine, thereby providing suggestion functions that reflect the user's latest interests and circumstances.

[0473] Hardware and software used

[0474] This system uses the following hardware and software:

[0475] Server: A central computing unit for data collection, preprocessing, running generative models, filtering keywords, and integrating them into suggestion functions. Specific examples include high-performance cloud servers (e.g., AWS, GCP).

[0476] User terminal: A device that generates communication data and displays suggestions. Examples include smartphones, tablets, and PCs.

[0477] Generative AI model: A generative model that uses natural language processing techniques. In particular, we use the latest generative AI models such as BERT and GPT.

[0478] Data processing and calculation

[0479] 1. Collecting Talk Data:

[0480] The server periodically collects chat data from the user's device. This process involves obtaining message data from chat apps such as the LINE application.

[0481] The collected data is stored in an encrypted database to ensure security.

[0482] 2. Data preprocessing:

[0483] The server retrieves the collected data from the encrypted database and removes unnecessary information (e.g. stamps, images, videos).

[0484] The preprocessed data is split into text sentences and converted into a clean format, removing special characters and unnecessary spaces in the process.

[0485] 3. Data input to the generative model:

[0486] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4), which generates potential search keywords from the user's conversation.

[0487] The generated keywords are output along with a confidence score.

[0488] 4. Keyword extraction and filtering:

[0489] The server runs the keywords obtained from the generative AI model through a filtering algorithm to remove inappropriate content and spam.

[0490] The filtered keywords are saved as a reliable list.

[0491] 5. Reflection in search suggestions:

[0492] The server uploads the filtered keywords to the search engine's suggestion database, which is updated in real time.

[0493] 6. Suggestions for users:

[0494] The user's device will display customized search suggestions through the browser or search application, allowing the user to efficiently access the information they need.

[0495] Specific examples

[0496] Some specific examples are given below.

[0497] Case 1: Movie talk

[0498] 1. User A and User B have a conversation about "Do you know what movies are out recently?"

[0499] 2. The server collects and preprocesses this talk data.

[0500] 3. The generative model generates keywords such as "recent movies" and "movies 2023."

[0501] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[0502] 5. When User A performs a web search, suggestions such as "recent movies" and "movies 2023" are displayed.

[0503] Case 2: Talking about the weather

[0504] 1. User C sends a message saying, "Do you know what the weather will be like tomorrow?"

[0505] 2. The server collects and preprocesses this message.

[0506] 3. The generative model generates keywords such as "tomorrow's weather" and "weather forecast."

[0507] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[0508] 5. When User C uses a search engine, keywords such as "tomorrow's weather" and "weather forecast" are suggested.

[0509] This mechanism makes it possible to provide effective suggestions based on the user's latest interests.

[0510] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0511] Step 1:

[0512] The server collects chat data from the user's device. Specifically, it obtains the message history of the chat application. For example, it collects message data from LINE such as "Do you know what movies are out these days?" The input is the user's communication data, and the output is the result of collecting this data. The data is encrypted and stored in a secure database.

[0513] Step 2:

[0514] The server preprocesses the collected chat data. The input is chat data read from an encrypted database, and the output is clean data with unnecessary information removed. Specifically, non-text information such as stamps, images, and videos in messages is removed, and the text is divided into sentences. Special characters and unnecessary spaces are also removed during this process.

[0515] Step 3:

[0516] The server inputs the preprocessed data into a generative model. The input is cleaned and formatted talk data, and the output is generated keywords and their confidence scores. Specifically, a generative AI model (e.g., GPT-4) is used to analyze the text and generate potential search keywords such as "recent movies" and "movies 2023."

[0517] Step 4:

[0518] The server inputs the keywords obtained from the generative model into a filtering algorithm. The input is the generated keywords, and the output is a filtered, more reliable list of keywords. Specific filtering actions include removing spam and inappropriate content. For example, if the keyword "recent movies" is appropriate, it will be left as is, but "spammy content" will be removed.

[0519] Step 5:

[0520] The server uploads the filtered keyword list to the search engine's suggestion database. The input is the filtered keyword list, and the output is a search suggestion function that integrates the list. Specific operations include writing to the database and updating it in real time. This process adds keywords such as "movies 2023" to the suggestion database.

[0521] Step 6:

[0522] When a user uses a search engine on their device, the generated keywords are displayed as suggestions in real time. The input is the user's search query, and the output is the keywords displayed in the suggestion box. Specifically, when a user begins to type in the search box, keywords such as "recent movies" and "weather forecast" are presented as candidates. As a result, users can efficiently access related information.

[0523] (Application example 1)

[0524] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0525] Conventional navigation systems are unable to understand the individual needs and preferences of users, and are limited to providing pre-set route information and destinations. This means that they are unable to appropriately suggest places and information that are truly of interest to users. In relation to this, there is a need for real-time, personalized navigation based on in-car conversations and behavior.

[0526] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0527] In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative AI model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into a navigation system, and means for displaying the keywords generated based on conversation data collected in the car on the navigation system. This enables real-time keyword generation based on conversation data in the car and personalized destination and information suggestions based on the generated keywords.

[0528] "User communications data" is information generated by users, such as conversations in the car or text messages.

[0529] "Preprocessing" refers to the process of removing noise and unnecessary information from collected user communication data and shaping the data to make it easier to analyze using a generative model.

[0530] A "generative model" is a model that extracts latent keywords from user communication data that has been preprocessed using natural language processing technology. Specifically, AI models such as BERT and GPT fall into this category.

[0531] "Integrating into the navigation system" refers to the process of incorporating the keywords extracted by the generative model into the database of the vehicle's navigation system, enabling information to be provided in real time.

[0532] A "navigation system" is an electronic system installed in a vehicle that allows the user to set a destination and provides optimal route guidance.

[0533] "Conversational data collected in the vehicle" refers to the user's voice and text information obtained through the vehicle's microphone and text input device.

[0534] "Keywords" are important words or phrases extracted from communication data by the generative model and used in navigation systems to identify specific information or destinations.

[0535] "Displaying" refers to visually providing information and destinations related to the extracted keywords to the user on the display of the navigation system.

[0536] A system for implementing this invention is a navigation system installed in an autonomous vehicle that generates potential keywords based on user communication data and uses them to provide personalized navigation guidance.

[0537] System Configuration

[0538] The system includes the following major components:

[0539] 1. In-vehicle server

[0540] 2. Generative Model

[0541] 3. Navigation system

[0542] 4. In-car displays

[0543] 5. Microphones and Text Input Devices

[0544] What the program does

[0545] In-vehicle server

[0546] The in-vehicle server collects conversation data through the vehicle's microphone and text input devices. The collected data is stored in an encrypted database. This collection process is performed periodically to obtain the latest data.

[0547] Pretreatment

[0548] The in-vehicle server preprocesses the collected conversation data into a format that is easy to analyze. Specifically, it removes noise and unnecessary information and divides the data into sentences. This preprocessing allows the generative model to effectively extract keywords.

[0549] Generative Model

[0550] The preprocessed conversation data is input to a generative model (e.g., BERT or GPT), which analyzes the data and extracts keywords that the user is likely to be interested in from the conversations in the car.

[0551] Integration into navigation systems

[0552] The in-vehicle server integrates the generated keywords into the navigation system, which then uses these customized keywords to display personalized routes and information when the user sets a destination.

[0553] Displaying suggestions to users

[0554] The in-car display will show customized suggestions through the navigation system, allowing users to easily find relevant information and enjoy greater convenience.

[0555] Hardware and software used

[0556] The system includes the following specific hardware and software:

[0557] In-vehicle server: A server installed in the vehicle for collecting and analyzing data

[0558] Microphone and text input device: A device for collecting voice data and text information inside the vehicle.

[0559] Generative models: AI models that use natural language processing techniques such as BERT and GPT

[0560] Navigation system: In-vehicle route guidance and information display system

[0561] Encryption protocol: Technology to ensure security of data communication such as TLS / SSL

[0562] Examples and prompts

[0563] Specific examples

[0564] Case 1: A user asks in the car, "Is there a good ramen restaurant nearby?"

[0565] An in-vehicle server collects and pre-processes this data.

[0566] GPT-4 extracts keywords such as "delicious ramen restaurant" and "recommended ramen restaurant."

[0567] The extracted keywords are integrated into the navigation system.

[0568] When the user launches the navigation system, suggestions such as "delicious ramen restaurants" are displayed, and navigation can be started immediately.

[0569] Prompt Sentence Examples

[0570] Generate keywords that will attract users' interest based on the latest in-car conversation data. Example keywords are shown below.

[0571] "Delicious ramen restaurant"

[0572] "Nearby gas station"

[0573] "Playground for children"

[0574] In this way, the system of the present invention can improve the convenience and personalized experience of autonomous vehicles by utilizing the user's real-time communication data and enhancing the navigation system's suggestion functions.

[0575] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0576] Step 1:

[0577] The server collects the user's communication data (conversation data and text messages) from the microphone and text input device. The input is the user's real-time conversation data, and the output is formatted data stored in an encrypted database. This collection process is performed periodically to capture the latest communication data occurring inside the vehicle.

[0578] Step 2:

[0579] The server preprocesses the collected conversation data. Specifically, it removes noise and unnecessary information and divides it into sentences. The input is raw data from an encrypted database, and the output is noise-removed, formatted text data. This preprocessing allows the generative model to effectively extract keywords.

[0580] Step 3:

[0581] The server inputs the preprocessed conversation data into a generative model (e.g., GPT-4 or BERT). The generative model analyzes the conversation data and extracts keywords of potential interest. The input is preprocessed text data, and the output is a list of keywords as the analysis result.

[0582] Step 4:

[0583] The server integrates the generated keywords into the navigation system. Specifically, it adds the generated keywords to the navigation system database or displays them as real-time search suggestions. The input is the generated keyword list, and the output is the suggestion data integrated into the navigation system.

[0584] Step 5:

[0585] The navigation system will suggest destinations and related information to the user based on the generated keywords through the in-car display. The input is the suggestion data integrated into the navigation system, and the output is customized destination and route information displayed on the display. This step allows the user to intuitively browse customized destination information.

[0586] Step 6:

[0587] When the user selects a suggested keyword on the in-car display, the navigation system generates the optimal route based on the selected destination and information and begins providing guidance. The input is the user's selected keyword, and the output is the specific guided route.

[0588] This series of processing steps allows users to enjoy personalized navigation based on the conversations they have in the car, greatly improving convenience.

[0589] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0590] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines an emotion engine that recognizes the user's emotions to provide more appropriate search suggestions based on the user's emotional state. This makes it possible to further personalize the user experience and improve satisfaction.

[0591] What the program does

[0592] 1. Collecting chat data

[0593] The server collects chat data from user devices and stores it in encrypted form in a secure database. This collection can occur in real time or periodically.

[0594] 2. Data Preprocessing

[0595] The server preprocesses the collected chat data, specifically filtering out unnecessary information (e.g., stickers, images, videos) and formatting it into a format suitable for the generative model.

[0596] 3. Data input to the generative model

[0597] The server inputs the preprocessed chat data into a generative model (e.g., BERT or GPT), which analyzes this data and extracts keywords that potentially interest the user.

[0598] 4. Introducing the Emotion Engine

[0599] The server inputs the same data into the emotion engine in parallel with the conversation data. The emotion engine uses natural language processing technology to recognize emotions from the user's conversation.

[0600] 5. Keyword extraction and sentiment consideration

[0601] The generative AI model analyzes the chat data to generate important keywords, and the emotion engine weights the keywords based on the user's emotional data. For example, if the user is expressing positive emotions, it will prioritize keywords that match that mood.

[0602] 6. Keyword Filtering

[0603] The server filters the generated keywords to remove inappropriate content and spam, a process that also takes sentiment data into account.

[0604] 7. Reflection in suggestion function

[0605] The server then integrates the extracted keywords into the search engine's suggestion function, taking into account emotional data. Specifically, it uses the Yahoo! Search API to upload appropriate keywords as suggestion data.

[0606] 8. Displaying suggestions to users

[0607] When a user uses a search engine on their device, they will see personalized search suggestions based on their emotions, helping them quickly find the information that best matches their search intent.

[0608] Specific examples

[0609] Case 1: Movie talk

[0610] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[0611] 2. The server collects and preprocesses this talk data.

[0612] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[0613] 4. The emotion engine recognizes user A's positive emotions and weights positive movie keywords that match those emotions.

[0614] 5. The server integrates these keywords into the search engine's suggestion function, and when User A uses Yahoo Search, emotion-based suggestions such as "recent movies" and "movies 2023" are displayed.

[0615] Case 2: Talking about the weather

[0616] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[0617] 2. The server collects and preprocesses this message.

[0618] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[0619] 4. The emotion engine recognizes user C's anxious emotion and weights keywords for detailed weather information that correspond to that emotion.

[0620] 5. The server integrates these keywords into the search engine's suggestion function, and when User C uses Yahoo Search, detailed suggestions such as "tomorrow's weather" and "weather forecast" based on emotions are displayed.

[0621] Privacy and Data Security

[0622] During the data collection and analysis process, the server anonymizes the collected data to protect user privacy and makes it impossible to identify individuals. In addition, encryption protocols (e.g., TLS / SSL) are used for data transmission.

[0623] Users can opt out of data collection, sentiment analysis, and suggestion functions from the settings screen, allowing them to use the service in a privacy-conscious manner.

[0624] The present invention significantly improves user experience by providing search suggestions that take into account the user's communication data and their sentiments.

[0625] The processing flow will be explained below.

[0626] Step 1:

[0627] The server collects chat data from user devices, periodically retrieves the data via API, and stores it in a secure database. The data is protected using AES encryption technology when stored.

[0628] Step 2:

[0629] The server preprocesses the collected chat data. Specifically, it filters unnecessary information (e.g., stamps, images, videos) from the text data, divides it into chats, and formats it appropriately.

[0630] Step 3:

[0631] The server inputs the preprocessed talk data into a generative model (e.g., a natural language processing model such as BERT or GPT), which analyzes the talk data and extracts important keywords.

[0632] Step 4:

[0633] The server also inputs the chat data into the emotion engine, which uses natural language processing technology to analyze and recognize emotions from the user's text. The analysis results are output as emotion labels such as anger, joy, and sadness.

[0634] Step 5:

[0635] The generative model generates important keywords based on the preprocessed chat data. At this time, the emotion engine references the emotional data recognized and weights the keywords. For example, if the user is expressing positive emotions, it will prioritize keywords related to happy content.

[0636] Step 6:

[0637] The server-generated keywords are filtered to remove inappropriate content and spam from the keyword list and select only reliable keywords. Sentiment data is also taken into account in this filtering process.

[0638] Step 7:

[0639] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, the keywords are uploaded as suggestion data via the search engine's API. This allows emotion-sensitive keywords to be displayed as suggestions in the search box.

[0640] Step 8:

[0641] When a user uses a search engine on their device, search suggestions customized based on their emotions are displayed. When a user types text into the search box, keywords integrated by the server are displayed as suggestions in real time.

[0642] Step 9:

[0643] Users can click on suggested keywords and perform a search to quickly retrieve relevant information, which is a process that best matches the user's search intent and increases user satisfaction.

[0644] Step 10:

[0645] The server collects data on users' search behavior and uses it to improve the system, such as which suggestions were clicked and which search results were viewed, and uses this data to train the generative model and emotion engine.

[0646] Step 11:

[0647] The generative model is retrained based on the collected feedback data to improve its keyword extraction accuracy the next time, which allows it to provide more relevant suggestions to users.

[0648] Step 12:

[0649] Users can opt out of data collection, sentiment analysis, and suggestion functions through the settings screen. This protects users' privacy and provides an environment where users can use the service with peace of mind.

[0650] Example 2

[0651] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] The suggestion function of conventional search engines presented keywords without taking into account the user's emotions or the content of their communication data, which meant that they were unable to fully meet the needs of users. In addition, they lacked privacy protection, making it difficult to provide an environment where users could use the search engine with peace of mind.

[0653] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for weighting the generated search keywords based on the user's emotion data, means for filtering the generated keywords to remove inappropriate content and spam, and means for integrating the generated search keywords into the suggestion function of the search engine. This makes it possible to provide personalized search suggestions that reflect the user's emotions and the content of the communication data, while also ensuring privacy protection.

[0654] "Communications Data" refers to information such as text messages, audio data, and video data that a user sends or receives.

[0655] "Preprocessing" is the process of removing unnecessary information from collected communication data and converting it into a format suitable for analysis and processing.

[0656] A "generative model" is an algorithm or machine learning model that uses natural language processing technology to extract potential search keywords from preprocessed communication data.

[0657] "Emotion data" is data that indicates a user's emotions and psychological state, recognized from communication data using natural language processing technology.

[0658] "Weighting" is the process of assigning a certain score or priority to the generated keywords to adjust their importance.

[0659] "Filtering" is the process of removing inappropriate content and spam from the generated keywords.

[0660] A "suggestion function" is a function that automatically suggests related keywords in response to a query entered by a user in a search engine.

[0661] "Privacy protection" refers to the means and technologies that anonymize users' personal information and communication data and protect them from leaks to third parties.

[0662] "Encryption" is the technology of transforming data using specific algorithms so that only authorized persons can decipher it.

[0663] "Data collection" is the process of importing user communication data via a network into a server or database.

[0664] "Natural language processing" is the field of technology that enables computers to understand, generate, and manipulate human language.

[0665] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines this with an emotion engine that recognizes the user's emotions, to provide more appropriate search suggestions based on the user's emotional state.

[0666] The server first collects user communication data. This collected data is then stored in a secure database in encrypted form using AES 256-bit encryption. The collected data is then sent to the server in real time or periodically.

[0667] The server then pre-processes the collected communication data. This involves filtering out media data such as stamps, images, and videos from the text. This data is then tokenized using a morphological analysis tool such as MeCab, and converted into a format suitable for analysis and processing.

[0668] The preprocessed data is then input into a generative model, such as BERT or GPT, which is implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model understands the context of the chat data and extracts keywords that potentially interest users.

[0669] In parallel, the server also inputs the same chat data into the emotion engine, which uses natural language processing (NLP) to recognize the user's emotions from the communication data. Emotions are classified into labels such as positive, negative, and neutral. Tools such as TextBlob and VADER are used for this emotion analysis.

[0670] The keywords extracted by the generative AI model are then weighted based on the emotional data recognized by the emotion engine. The weighting is done using a scoring algorithm to prioritize keywords that match a user's positive emotion.

[0671] The generated keywords then go through a filtering process, which removes inappropriate content and spam. This includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to remove inappropriate keywords, improving the quality of the user experience.

[0672] The filtered keywords are finally integrated into the search engine's suggestion function. Appropriate keywords are uploaded as suggestion data using the Yahoo! Search API, etc. This updates the suggestion database in real time.

[0673] When users use search engines, they will see personalized search suggestions, which will give users search suggestions optimized based on sentiment. For example, if you search for "recent movies," you will see relevant suggestions such as "movies 2023" and "trending movies."

[0674] Specific examples include the following cases:

[0675] Case 1: Movie talk

[0676] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[0677] 2. The server collects this chat data, encrypts it, and stores it in a database.

[0678] 3. The server preprocesses and tokenizes the text data.

[0679] 4. The generative model extracts keywords such as "recent movies" and "movies 2023" from the preprocessed data.

[0680] 5. The emotion engine recognizes User A's positive emotions and weights positive movie keywords.

[0681] 6. The server filters these keywords and removes inappropriate content.

[0682] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[0683] 8. When User A uses a search engine, suggestions such as "recent movies" and "movies 2023" appear.

[0684] Case 2: Talking about the weather

[0685] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[0686] 2. The server collects this message, encrypts it, and stores it in a database.

[0687] 3. The server preprocesses and tokenizes the text data.

[0688] 4. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast" from the preprocessed data.

[0689] 5. The emotion engine recognizes user C's anxious emotions and weights keywords for detailed weather information.

[0690] 6. The server filters these keywords and removes inappropriate content.

[0691] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[0692] 8. When User C uses a search engine, they will see emotion-based suggestions such as "tomorrow's weather" and "weather forecast."

[0693] An example of a prompt is:

[0694] "Are there any good movies out recently?"

[0695] "What's the weather going to be like tomorrow?"

[0696] In this way, the present invention can efficiently provide personalized search suggestions that take into account the user's communication data and emotions.

[0697] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0698] Step 1:

[0699] The server collects communication data from the user's device. Specifically, when a user chats using a messaging app, the text data is sent to the server via the network. At this time, the collected communication data is stored in a database using AES 256-bit encryption. The input is a text message from the user, and the output is encrypted communication data.

[0700] Step 2:

[0701] The server preprocesses the collected communication data. Specifically, it filters non-text information such as stamps, images, and videos from the text data, and divides the text into words using tokenization and morphological analysis tools (e.g., MeCab). The input to this step is the encrypted and stored communication data, and the output is clean text data after preprocessing.

[0702] Step 3:

[0703] The server inputs the preprocessed text data into a generative model. Specifically, it applies generative models such as BERT or GPT, which are implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model analyzes and extracts keywords that potentially interest the user from the preprocessed data. The input is the preprocessed text data, and the output is the extracted latent keywords.

[0704] Step 4:

[0705] The server simultaneously inputs the preprocessed text data into the emotion engine. The emotion engine uses natural language processing technology to recognize user emotions from the communication data. Specifically, it uses tools such as TextBlob and VADER to assign emotion labels such as positive, negative, and neutral to the data. The input is the preprocessed text data, and the output is the assigned emotion label.

[0706] Step 5:

[0707] The generative AI model weights the extracted keywords based on emotional data obtained from the emotion engine. For example, if a user expresses a positive emotion, the scoring algorithm is used to prioritize keywords that match that mood. The inputs are the extracted keywords and emotion labels, and the output is the weighted keywords.

[0708] Step 6:

[0709] The server filters the generated keywords to remove inappropriate content and spam. This process includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to ensure a good user experience. The input is weighted keywords, and the output is clean, filtered keywords.

[0710] Step 7:

[0711] The server finally integrates the filtered keywords into the search engine's suggestion function. Specifically, by uploading appropriate keywords as suggestion data using the search engine API, the suggestion function database is updated in real time. The input is the filtered keywords, and the output is the updated search engine's suggestion data.

[0712] Step 8:

[0713] When using a search engine, a user's device displays customized search suggestions, allowing the user to obtain search results optimized based on sentiment. For example, when searching for "recent movies," relevant suggestions such as "movies 2023" and "trending movies" are displayed. The input is a search query, and the output is search suggestions.

[0714] This allows users to experience personalized search suggestions based on emotions.

[0715] (Application example 2)

[0716] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0717] Conventional search engine suggestion functions are limited to suggesting keywords based on the user's communication data and do not take into account the user's emotional state, making it impossible to provide a truly personalized experience. In addition, user privacy may not be adequately protected, and ensuring safety is also an issue.

[0718] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into the search engine's suggestion function, means for recognizing the user's emotional state from the user's communication data, and means for weighting the generated search keywords based on the recognized emotional state. This makes it possible to suggest search keywords based on the user's emotional state, providing a more personalized user experience. Privacy protection is also ensured by anonymizing the communication data and emotional data.

[0719] "User Communication Data" means all data sent or received by a User through a Communication Device, including, in particular, text messages, voice data, and other information.

[0720] "Preprocessing" refers to a series of processes that remove unnecessary information from collected communication data and convert it into a format suitable for analysis.

[0721] A "generative model" refers to a machine learning algorithm that extracts and generates keywords that may be of interest to users based on collected and preprocessed data.

[0722] "Search engine suggestion function" refers to the function that predicts and displays related search keywords when a user types something into a search box.

[0723] "Means for recognizing emotional states" refers to algorithms that analyze a user's communication data and identify their emotional state (such as joy, sadness, or anxiety).

[0724] "Weighting" refers to the process of prioritizing generated keywords based on emotional state.

[0725] "Anonymization" refers to the process of removing personally identifiable information from collected data to protect user privacy.

[0726] The present invention relates to a system for generating search keywords based on communication data and emotional state of a user in a virtual store, and for making more personalized product suggestions. The system includes the following means.

[0727] A means of collecting and preprocessing user communication data

[0728] The server collects user communication data in real time or periodically via smartphones or head-mounted displays. The collected communication data is stored in an encrypted database (e.g., MySQL) on the server. Data security is ensured using encryption protocols such as TLS / SSL. Preprocessing involves filtering out unnecessary information (e.g., stamps, images, and videos) and converting text data into an appropriate format. This process makes the data easier to analyze.

[0729] A method using a generative model to generate potential search keywords from preprocessed communication data

[0730] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to extract search keywords that the user is likely to be interested in. This generative model can generate appropriate keywords based on past data and user behavior patterns.

[0731] A means to integrate generated search keywords into search engine suggestion functions

[0732] The extracted search keywords are filtered by the server to remove inappropriate content and spam, and then uploaded to the suggestion system (e.g., search engine API) within the virtual store and displayed when a user searches.

[0733] A means of recognizing a user's emotional state from their communication data

[0734] The server inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The emotional state is classified into major emotion categories such as joy, sadness, and anxiety. The recognition results play an important role in the subsequent keyword generation process.

[0735] A means of weighting generated search keywords based on perceived emotional states

[0736] The server adjusts the importance of generated search keywords based on the emotional data obtained from the emotion engine. For example, if a user feels like relaxing, keywords such as "relaxation goods" and "aroma" will be prioritized.

[0737] Specific examples

[0738] Specific scenarios

[0739] 1. User A searches for "relaxing products" in a virtual store.

[0740] 2. A smartphone app collects the user's relaxed emotional state along with their conversation data.

[0741] 3. The server preprocesses the data and inputs it into GPT-3 and the Microsoft Azure Emotion API.

[0742] 4. GPT-3 extracts keywords such as "relaxation," "relaxation goods," and "aroma."

[0743] 5. The Microsoft Azure Emotion API recognizes relaxed emotions.

[0744] 6. The suggestion system weights "relaxation," "relaxation goods," and "aroma" and integrates them into the virtual store's suggestion list.

[0745] 7. When User A browses the virtual store, products such as "relaxation goods" and "aroma" are displayed preferentially.

[0746] Prompt Sentence Examples

[0747] Collect chat data that shows users feeling relaxed and analyze it using an emotion engine. The generative AI model should extract keywords that users are likely to be interested in, such as "relaxation," "aroma," and "relaxation goods," and integrate these into the suggestion system.

[0748] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0749] Step 1:

[0750] The server collects the user's communication data from the smartphone or head-mounted display. The collected communication data is encrypted and stored in a database. The input is the user's communication data (text messages, voice data, and other information), and the output is the data stored in the database in encrypted form. This step includes specific operations to ensure the security of the data using the TLS / SSL protocol.

[0751] Step 2:

[0752] The server preprocesses the collected communication data. Specifically, it filters out unnecessary information (e.g., stamps, images, videos) and formats it as text data. The input is encrypted communication data, and the output is preprocessed text data. In this step, the data cleansing and filtering process is performed.

[0753] Step 3:

[0754] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to generate search keywords that the user is likely to be interested in. The input is the preprocessed text data, and the output is the generated search keywords. This step includes the specific operation of extracting keywords using a machine learning algorithm.

[0755] Step 4:

[0756] The server simultaneously inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The input is preprocessed text data, and the output is the recognized emotional state. In this step, natural language processing techniques are used to classify the user's emotion.

[0757] Step 5:

[0758] The server combines the search keywords obtained from the generative AI model with the emotional data obtained from the emotion engine and weights the keywords. The input is the generated search keywords and the recognized emotional state, and the output is the weighted search keywords. In this step, an algorithm is run to determine the priority of the keywords.

[0759] Step 6:

[0760] The server filters the weighted search keywords to remove inappropriate content and spam. The input is the weighted search keywords, and the output is the filtered keywords. This step includes specific operations to remove inappropriate keywords using text analysis techniques.

[0761] Step 7:

[0762] The server integrates the filtered keywords into the search engine's suggestion function. The input is the filtered keywords, and the output is the keywords uploaded as search engine suggestion data. In this step, the search engine API is used to add the keywords to the suggestion system.

[0763] Step 8:

[0764] When a user uses a virtual store, search suggestions customized based on their emotions are displayed. The input is the search engine's suggestion data, and the output is the customized suggestions displayed on the user's device. This step includes the specific operation of displaying appropriate suggestions through the user interface.

[0765] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0766] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0767] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0768] [Third embodiment]

[0769] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0770] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0771] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0772] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0773] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0774] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0775] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0776] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0777] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0778] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0779] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0780] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0781] This invention is a system that collects and analyzes user communication data and integrates it into the suggestion function of a search engine. This system can effectively suggest information that users have forgotten about or that they potentially want to know.

[0782] What the program does

[0783] 1. Collecting chat data

[0784] The server collects chat data from users' devices and stores it in an encrypted database. This collection process is performed periodically, allowing users to view their most recent chat data.

[0785] 2. Data Preprocessing

[0786] The server formats the collected chat data into a format suitable for the generative model. Specifically, it removes unnecessary information (e.g., stamps, images, and videos) from the text data and divides it into sentences. This process allows the generative model to effectively extract keywords.

[0787] 3. Data input to the generative model

[0788] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT and GPT. The model analyzes the chat data and extracts keywords that users are likely to be interested in.

[0789] 4. Keyword extraction and filtering

[0790] A generative model generates important keywords from the chat data, which are then further filtered to remove spam and inappropriate content, resulting in highly reliable keywords.

[0791] 5. Reflection in search suggestions

[0792] The server integrates the extracted keywords into the search engine's suggestion function, specifically by uploading the generated keywords as search suggestion data, which are displayed when a user starts typing in the search box.

[0793] 6. Displaying suggestions to users

[0794] When a user uses a search engine on their device, customized search suggestions are displayed, allowing the user to easily find relevant information.

[0795] Specific examples

[0796] Case 1: Movie talk

[0797] 1. User A and User B have a conversation on LINE about "Do you know what movies are out recently?"

[0798] 2. The server collects and pre-processes this data.

[0799] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[0800] 4. The server integrates these keywords into the search engine's suggestion function.

[0801] 5. When User A uses Yahoo Search, suggestions such as "Recent Movies" and "Movies 2023" are displayed, and the user can click on them to search for more information.

[0802] Case 2: Talking about the weather

[0803] 1. User C sends a message on LINE saying, "Do you know what the weather will be like tomorrow?"

[0804] 2. The server collects and preprocesses this message.

[0805] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[0806] 4. The server integrates these keywords into the search engine's suggestion function.

[0807] 5. When User C uses Yahoo Search, keywords such as "tomorrow's weather" and "weather forecast" are displayed in the suggestion box, allowing them to search directly.

[0808] Privacy and Data Security

[0809] To protect user privacy during data collection and analysis, the server anonymizes the collected data so that it cannot be used to identify individuals, and uses encryption protocols (e.g., TLS / SSL) for data transmission.

[0810] Users can opt out of this suggestion feature from the settings screen, providing peace of mind regarding their privacy.

[0811] In this way, the present invention provides a system that utilizes users' communication data to understand their latent search intent and reflect this in the search engine's suggestion function, thereby facilitating users' search activities.

[0812] The processing flow will be explained below.

[0813] Step 1:

[0814] The server collects chat data from user devices, retrieves it in real time or periodically via API, and stores it in a secure database. The data is protected using encryption technology while stored.

[0815] Step 2:

[0816] The server preprocesses the collected talk data into a format suitable for the generative model. Specifically, unnecessary information such as stamps, images, and videos are filtered from the text data, and the data is divided into talks and formatted as text tokens.

[0817] Step 3:

[0818] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT or GPT. This model analyzes the chat data and extracts search keywords that the user is likely to be interested in.

[0819] Step 4:

[0820] The generative model extracts keywords from the analysis of the chat data. Specifically, it selects and lists highly important keywords and phrases. These keywords then undergo a filtering process to remove inappropriate content and spam.

[0821] Step 5:

[0822] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, it uploads the generated keywords as suggestion data through the Yahoo! Search API. This upload process immediately updates the keywords to the suggestion list.

[0823] Step 6:

[0824] When a user uses a search engine on their device, the search engine displays customized search suggestions. As the user types in the search box, the keywords integrated in step 5 are displayed in real time.

[0825] Step 7:

[0826] Users can click on the suggested keywords and perform a search to retrieve related information, allowing them to quickly access the information they were potentially looking for.

[0827] Step 8:

[0828] The server collects data on users' search behavior and uses it to improve the system. Specifically, it collects data such as which suggestions users clicked and which search results they viewed, and reuses this data as training data for the generative model.

[0829] Step 9:

[0830] The generative model is retrained based on the collected feedback data, which improves the accuracy of keyword extraction the next time and allows the model to provide more appropriate suggestions to users.

[0831] Step 10:

[0832] Users can opt out of data collection and suggestion features in the settings screen, allowing them to use the service in a privacy-conscious manner.

[0833] Example 1

[0834] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0835] The suggestion functions of conventional search engines are based on users' past search history and general trends, and therefore often fail to effectively provide keywords that are relevant to the user's current interests and circumstances. Furthermore, from the perspective of privacy protection, there are sometimes concerns about whether user data is being managed appropriately. Therefore, there is a need for the development of a system that reflects the information users are seeking in everyday conversations in real time while protecting their privacy.

[0836] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0837] In this invention, the server includes means for collecting and preprocessing user communication data, means for removing unnecessary information from the preprocessed communication data and dividing it into sentence units, means for inputting the divided communication data into a generative model to generate potential search keywords, and means for filtering the generated search keywords and integrating them into the search engine's suggestion function. This makes it possible to provide a suggestion function that reflects the user's latest interests and situation, thereby achieving both an improved user experience and privacy protection.

[0838] "Communication data" refers to text information exchanged by users using chat applications, messaging services, etc.

[0839] "Preprocessing" is the process of removing unnecessary information from collected communication data and formatting it into a format that is easy to analyze.

[0840] A "generative model" is an algorithm or software that uses natural language processing techniques to generate potential search keywords from preprocessed communication data.

[0841] A "search engine suggestion function" is a search assistance function that automatically suggests related keywords when a user begins typing in the search box.

[0842] "Filtering" is the process of removing inappropriate content and spam from the generated keywords, leaving only reliable keywords.

[0843] An "encrypted database" is a database that is protected using encryption technology to ensure data security.

[0844] "Anonymization" is a process that protects privacy by removing personally identifiable information from collected communications data.

[0845] The present invention is a system that collects and preprocesses user communication data, and then integrates keywords generated from that data into the suggestion function of a search engine, thereby providing suggestion functions that reflect the user's latest interests and circumstances.

[0846] Hardware and software used

[0847] This system uses the following hardware and software:

[0848] Server: A central computing unit for data collection, preprocessing, running generative models, filtering keywords, and integrating them into suggestion functions. Specific examples include high-performance cloud servers (e.g., AWS, GCP).

[0849] User terminal: A device that generates communication data and displays suggestions. Examples include smartphones, tablets, and PCs.

[0850] Generative AI model: A generative model that uses natural language processing techniques. In particular, we use the latest generative AI models such as BERT and GPT.

[0851] Data processing and calculation

[0852] 1. Collecting Talk Data:

[0853] The server periodically collects chat data from the user's device. This process involves obtaining message data from chat apps such as the LINE application.

[0854] The collected data is stored in an encrypted database to ensure security.

[0855] 2. Data preprocessing:

[0856] The server retrieves the collected data from the encrypted database and removes unnecessary information (e.g. stamps, images, videos).

[0857] The preprocessed data is split into text sentences and converted into a clean format, removing special characters and unnecessary spaces in the process.

[0858] 3. Data input to the generative model:

[0859] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4), which generates potential search keywords from the user's conversation.

[0860] The generated keywords are output along with a confidence score.

[0861] 4. Keyword extraction and filtering:

[0862] The server runs the keywords obtained from the generative AI model through a filtering algorithm to remove inappropriate content and spam.

[0863] The filtered keywords are saved as a reliable list.

[0864] 5. Reflection in search suggestions:

[0865] The server uploads the filtered keywords to the search engine's suggestion database, which is updated in real time.

[0866] 6. Suggestions for users:

[0867] The user's device will display customized search suggestions through the browser or search application, allowing the user to efficiently access the information they need.

[0868] Specific examples

[0869] Some specific examples are given below.

[0870] Case 1: Movie talk

[0871] 1. User A and User B have a conversation about "Do you know what movies are out recently?"

[0872] 2. The server collects and preprocesses this talk data.

[0873] 3. The generative model generates keywords such as "recent movies" and "movies 2023."

[0874] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[0875] 5. When User A performs a web search, suggestions such as "recent movies" and "movies 2023" are displayed.

[0876] Case 2: Talking about the weather

[0877] 1. User C sends a message saying, "Do you know what the weather will be like tomorrow?"

[0878] 2. The server collects and preprocesses this message.

[0879] 3. The generative model generates keywords such as "tomorrow's weather" and "weather forecast."

[0880] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[0881] 5. When User C uses a search engine, keywords such as "tomorrow's weather" and "weather forecast" are suggested.

[0882] This mechanism makes it possible to provide effective suggestions based on the user's latest interests.

[0883] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0884] Step 1:

[0885] The server collects chat data from the user's device. Specifically, it obtains the message history of the chat application. For example, it collects message data from LINE such as "Do you know what movies are out these days?" The input is the user's communication data, and the output is the result of collecting this data. The data is encrypted and stored in a secure database.

[0886] Step 2:

[0887] The server preprocesses the collected chat data. The input is chat data read from an encrypted database, and the output is clean data with unnecessary information removed. Specifically, non-text information such as stamps, images, and videos in messages is removed, and the text is divided into sentences. Special characters and unnecessary spaces are also removed during this process.

[0888] Step 3:

[0889] The server inputs the preprocessed data into a generative model. The input is cleaned and formatted talk data, and the output is generated keywords and their confidence scores. Specifically, a generative AI model (e.g., GPT-4) is used to analyze the text and generate potential search keywords such as "recent movies" and "movies 2023."

[0890] Step 4:

[0891] The server inputs the keywords obtained from the generative model into a filtering algorithm. The input is the generated keywords, and the output is a filtered, more reliable list of keywords. Specific filtering actions include removing spam and inappropriate content. For example, if the keyword "recent movies" is appropriate, it will be left as is, but "spammy content" will be removed.

[0892] Step 5:

[0893] The server uploads the filtered keyword list to the search engine's suggestion database. The input is the filtered keyword list, and the output is a search suggestion function that integrates the list. Specific operations include writing to the database and updating it in real time. This process adds keywords such as "movies 2023" to the suggestion database.

[0894] Step 6:

[0895] When a user uses a search engine on their device, the generated keywords are displayed as suggestions in real time. The input is the user's search query, and the output is the keywords displayed in the suggestion box. Specifically, when a user begins to type in the search box, keywords such as "recent movies" and "weather forecast" are presented as candidates. As a result, users can efficiently access related information.

[0896] (Application example 1)

[0897] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0898] Conventional navigation systems are unable to understand the individual needs and preferences of users, and are limited to providing pre-set route information and destinations. This means that they are unable to appropriately suggest places and information that are truly of interest to users. In relation to this, there is a need for real-time, personalized navigation based on in-car conversations and behavior.

[0899] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0900] In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative AI model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into a navigation system, and means for displaying the keywords generated based on conversation data collected in the car on the navigation system. This enables real-time keyword generation based on conversation data in the car and personalized destination and information suggestions based on the generated keywords.

[0901] "User communications data" is information generated by users, such as conversations in the car or text messages.

[0902] "Preprocessing" refers to the process of removing noise and unnecessary information from collected user communication data and shaping the data to make it easier to analyze using a generative model.

[0903] A "generative model" is a model that extracts latent keywords from user communication data that has been preprocessed using natural language processing technology. Specifically, AI models such as BERT and GPT fall into this category.

[0904] "Integrating into the navigation system" refers to the process of incorporating the keywords extracted by the generative model into the database of the vehicle's navigation system, enabling information to be provided in real time.

[0905] A "navigation system" is an electronic system installed in a vehicle that allows the user to set a destination and provides optimal route guidance.

[0906] "Conversational data collected in the vehicle" refers to the user's voice and text information obtained through the vehicle's microphone and text input device.

[0907] "Keywords" are important words or phrases extracted from communication data by the generative model and used in navigation systems to identify specific information or destinations.

[0908] "Displaying" refers to visually providing information and destinations related to the extracted keywords to the user on the display of the navigation system.

[0909] A system for implementing this invention is a navigation system installed in an autonomous vehicle that generates potential keywords based on user communication data and uses them to provide personalized navigation guidance.

[0910] System Configuration

[0911] The system includes the following major components:

[0912] 1. In-vehicle server

[0913] 2. Generative Model

[0914] 3. Navigation system

[0915] 4. In-car displays

[0916] 5. Microphones and Text Input Devices

[0917] What the program does

[0918] In-vehicle server

[0919] The in-vehicle server collects conversation data through the vehicle's microphone and text input devices. The collected data is stored in an encrypted database. This collection process is performed periodically to obtain the latest data.

[0920] Pretreatment

[0921] The in-vehicle server preprocesses the collected conversation data into a format that is easy to analyze. Specifically, it removes noise and unnecessary information and divides the data into sentences. This preprocessing allows the generative model to effectively extract keywords.

[0922] Generative Model

[0923] The preprocessed conversation data is input to a generative model (e.g., BERT or GPT), which analyzes the data and extracts keywords that the user is likely to be interested in from the conversations in the car.

[0924] Integration into navigation systems

[0925] The in-vehicle server integrates the generated keywords into the navigation system, which then uses these customized keywords to display personalized routes and information when the user sets a destination.

[0926] Displaying suggestions to users

[0927] The in-car display will show customized suggestions through the navigation system, allowing users to easily find relevant information and enjoy greater convenience.

[0928] Hardware and software used

[0929] The system includes the following specific hardware and software:

[0930] In-vehicle server: A server installed in the vehicle for collecting and analyzing data

[0931] Microphone and text input device: A device for collecting voice data and text information inside the vehicle.

[0932] Generative models: AI models that use natural language processing techniques such as BERT and GPT

[0933] Navigation system: In-vehicle route guidance and information display system

[0934] Encryption protocol: Technology to ensure security of data communication such as TLS / SSL

[0935] Examples and prompts

[0936] Specific examples

[0937] Case 1: A user asks in the car, "Is there a good ramen restaurant nearby?"

[0938] An in-vehicle server collects and pre-processes this data.

[0939] GPT-4 extracts keywords such as "delicious ramen restaurant" and "recommended ramen restaurant."

[0940] The extracted keywords are integrated into the navigation system.

[0941] When the user launches the navigation system, suggestions such as "delicious ramen restaurants" are displayed, and navigation can be started immediately.

[0942] Prompt Sentence Examples

[0943] Generate keywords that will attract users' interest based on the latest in-car conversation data. Example keywords are shown below.

[0944] "Delicious ramen restaurant"

[0945] "Nearby gas station"

[0946] "Playground for children"

[0947] In this way, the system of the present invention can improve the convenience and personalized experience of autonomous vehicles by utilizing the user's real-time communication data and enhancing the navigation system's suggestion functions.

[0948] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0949] Step 1:

[0950] The server collects the user's communication data (conversation data and text messages) from the microphone and text input device. The input is the user's real-time conversation data, and the output is formatted data stored in an encrypted database. This collection process is performed periodically to capture the latest communication data occurring inside the vehicle.

[0951] Step 2:

[0952] The server preprocesses the collected conversation data. Specifically, it removes noise and unnecessary information and divides it into sentences. The input is raw data from an encrypted database, and the output is noise-removed, formatted text data. This preprocessing allows the generative model to effectively extract keywords.

[0953] Step 3:

[0954] The server inputs the preprocessed conversation data into a generative model (e.g., GPT-4 or BERT). The generative model analyzes the conversation data and extracts keywords of potential interest. The input is preprocessed text data, and the output is a list of keywords as the analysis result.

[0955] Step 4:

[0956] The server integrates the generated keywords into the navigation system. Specifically, it adds the generated keywords to the navigation system database or displays them as real-time search suggestions. The input is the generated keyword list, and the output is the suggestion data integrated into the navigation system.

[0957] Step 5:

[0958] The navigation system will suggest destinations and related information to the user based on the generated keywords through the in-car display. The input is the suggestion data integrated into the navigation system, and the output is customized destination and route information displayed on the display. This step allows the user to intuitively browse customized destination information.

[0959] Step 6:

[0960] When the user selects a suggested keyword on the in-car display, the navigation system generates the optimal route based on the selected destination and information and begins providing guidance. The input is the user's selected keyword, and the output is the specific guided route.

[0961] This series of processing steps allows users to enjoy personalized navigation based on the conversations they have in the car, greatly improving convenience.

[0962] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0963] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines an emotion engine that recognizes the user's emotions to provide more appropriate search suggestions based on the user's emotional state. This makes it possible to further personalize the user experience and improve satisfaction.

[0964] What the program does

[0965] 1. Collecting chat data

[0966] The server collects chat data from user devices and stores it in encrypted form in a secure database. This collection can occur in real time or periodically.

[0967] 2. Data Preprocessing

[0968] The server preprocesses the collected chat data, specifically filtering out unnecessary information (e.g., stickers, images, videos) and formatting it into a format suitable for the generative model.

[0969] 3. Data input to the generative model

[0970] The server inputs the preprocessed chat data into a generative model (e.g., BERT or GPT), which analyzes this data and extracts keywords that potentially interest the user.

[0971] 4. Introducing the Emotion Engine

[0972] The server inputs the same data into the emotion engine in parallel with the conversation data. The emotion engine uses natural language processing technology to recognize emotions from the user's conversation.

[0973] 5. Keyword extraction and sentiment consideration

[0974] The generative AI model analyzes the chat data to generate important keywords, and the emotion engine weights the keywords based on the user's emotional data. For example, if the user is expressing positive emotions, it will prioritize keywords that match that mood.

[0975] 6. Keyword Filtering

[0976] The server filters the generated keywords to remove inappropriate content and spam, a process that also takes sentiment data into account.

[0977] 7. Reflection in suggestion function

[0978] The server then integrates the extracted keywords into the search engine's suggestion function, taking into account emotional data. Specifically, it uses the Yahoo! Search API to upload appropriate keywords as suggestion data.

[0979] 8. Displaying suggestions to users

[0980] When a user uses a search engine on their device, they will see personalized search suggestions based on their emotions, helping them quickly find the information that best matches their search intent.

[0981] Specific examples

[0982] Case 1: Movie talk

[0983] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[0984] 2. The server collects and preprocesses this talk data.

[0985] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[0986] 4. The emotion engine recognizes user A's positive emotions and weights positive movie keywords that match those emotions.

[0987] 5. The server integrates these keywords into the search engine's suggestion function, and when User A uses Yahoo Search, emotion-based suggestions such as "recent movies" and "movies 2023" are displayed.

[0988] Case 2: Talking about the weather

[0989] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[0990] 2. The server collects and preprocesses this message.

[0991] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[0992] 4. The emotion engine recognizes user C's anxious emotion and weights keywords for detailed weather information that correspond to that emotion.

[0993] 5. The server integrates these keywords into the search engine's suggestion function, and when User C uses Yahoo Search, detailed suggestions such as "tomorrow's weather" and "weather forecast" based on emotions are displayed.

[0994] Privacy and Data Security

[0995] During the data collection and analysis process, the server anonymizes the collected data to protect user privacy and makes it impossible to identify individuals. In addition, encryption protocols (e.g., TLS / SSL) are used for data transmission.

[0996] Users can opt out of data collection, sentiment analysis, and suggestion functions from the settings screen, allowing them to use the service in a privacy-conscious manner.

[0997] The present invention significantly improves user experience by providing search suggestions that take into account the user's communication data and their sentiments.

[0998] The processing flow will be explained below.

[0999] Step 1:

[1000] The server collects chat data from user devices, periodically retrieves the data via API, and stores it in a secure database. The data is protected using AES encryption technology when stored.

[1001] Step 2:

[1002] The server preprocesses the collected chat data. Specifically, it filters unnecessary information (e.g., stamps, images, videos) from the text data, divides it into chats, and formats it appropriately.

[1003] Step 3:

[1004] The server inputs the preprocessed talk data into a generative model (e.g., a natural language processing model such as BERT or GPT), which analyzes the talk data and extracts important keywords.

[1005] Step 4:

[1006] The server also inputs the chat data into the emotion engine, which uses natural language processing technology to analyze and recognize emotions from the user's text. The analysis results are output as emotion labels such as anger, joy, and sadness.

[1007] Step 5:

[1008] The generative model generates important keywords based on the preprocessed chat data. At this time, the emotion engine references the emotional data recognized and weights the keywords. For example, if the user is expressing positive emotions, it will prioritize keywords related to happy content.

[1009] Step 6:

[1010] The server-generated keywords are filtered to remove inappropriate content and spam from the keyword list and select only reliable keywords. Sentiment data is also taken into account in this filtering process.

[1011] Step 7:

[1012] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, the keywords are uploaded as suggestion data via the search engine's API. This allows emotion-sensitive keywords to be displayed as suggestions in the search box.

[1013] Step 8:

[1014] When a user uses a search engine on their device, search suggestions customized based on their emotions are displayed. When a user types text into the search box, keywords integrated by the server are displayed as suggestions in real time.

[1015] Step 9:

[1016] Users can click on suggested keywords and perform a search to quickly retrieve relevant information, which is a process that best matches the user's search intent and increases user satisfaction.

[1017] Step 10:

[1018] The server collects data on users' search behavior and uses it to improve the system, such as which suggestions were clicked and which search results were viewed, and uses this data to train the generative model and emotion engine.

[1019] Step 11:

[1020] The generative model is retrained based on the collected feedback data to improve its keyword extraction accuracy the next time, which allows it to provide more relevant suggestions to users.

[1021] Step 12:

[1022] Users can opt out of data collection, sentiment analysis, and suggestion functions through the settings screen. This protects users' privacy and provides an environment where users can use the service with peace of mind.

[1023] Example 2

[1024] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1025] The suggestion function of conventional search engines presented keywords without taking into account the user's emotions or the content of their communication data, which meant that they were unable to fully meet the needs of users. In addition, they lacked privacy protection, making it difficult to provide an environment where users could use the search engine with peace of mind.

[1026] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for weighting the generated search keywords based on the user's emotion data, means for filtering the generated keywords to remove inappropriate content and spam, and means for integrating the generated search keywords into the suggestion function of the search engine. This makes it possible to provide personalized search suggestions that reflect the user's emotions and the content of the communication data, while also ensuring privacy protection.

[1027] "Communications Data" refers to information such as text messages, audio data, and video data that a user sends or receives.

[1028] "Preprocessing" is the process of removing unnecessary information from collected communication data and converting it into a format suitable for analysis and processing.

[1029] A "generative model" is an algorithm or machine learning model that uses natural language processing technology to extract potential search keywords from preprocessed communication data.

[1030] "Emotion data" is data that indicates a user's emotions and psychological state, recognized from communication data using natural language processing technology.

[1031] "Weighting" is the process of assigning a certain score or priority to the generated keywords to adjust their importance.

[1032] "Filtering" is the process of removing inappropriate content and spam from the generated keywords.

[1033] A "suggestion function" is a function that automatically suggests related keywords in response to a query entered by a user in a search engine.

[1034] "Privacy protection" refers to the means and technologies that anonymize users' personal information and communication data and protect them from leaks to third parties.

[1035] "Encryption" is the technology of transforming data using specific algorithms so that only authorized persons can decipher it.

[1036] "Data collection" is the process of importing user communication data via a network into a server or database.

[1037] "Natural language processing" is the field of technology that enables computers to understand, generate, and manipulate human language.

[1038] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines this with an emotion engine that recognizes the user's emotions, to provide more appropriate search suggestions based on the user's emotional state.

[1039] The server first collects user communication data. This collected data is then stored in a secure database in encrypted form using AES 256-bit encryption. The collected data is then sent to the server in real time or periodically.

[1040] The server then pre-processes the collected communication data. This involves filtering out media data such as stamps, images, and videos from the text. This data is then tokenized using a morphological analysis tool such as MeCab, and converted into a format suitable for analysis and processing.

[1041] The preprocessed data is then input into a generative model, such as BERT or GPT, which is implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model understands the context of the chat data and extracts keywords that potentially interest users.

[1042] In parallel, the server also inputs the same chat data into the emotion engine, which uses natural language processing (NLP) to recognize the user's emotions from the communication data. Emotions are classified into labels such as positive, negative, and neutral. Tools such as TextBlob and VADER are used for this emotion analysis.

[1043] The keywords extracted by the generative AI model are then weighted based on the emotional data recognized by the emotion engine. The weighting is done using a scoring algorithm to prioritize keywords that match a user's positive emotion.

[1044] The generated keywords then go through a filtering process, which removes inappropriate content and spam. This includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to remove inappropriate keywords, improving the quality of the user experience.

[1045] The filtered keywords are finally integrated into the search engine's suggestion function. Appropriate keywords are uploaded as suggestion data using the Yahoo! Search API, etc. This updates the suggestion database in real time.

[1046] When users use search engines, they will see personalized search suggestions, which will give users search suggestions optimized based on sentiment. For example, if you search for "recent movies," you will see relevant suggestions such as "movies 2023" and "trending movies."

[1047] Specific examples include the following cases:

[1048] Case 1: Movie talk

[1049] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[1050] 2. The server collects this chat data, encrypts it, and stores it in a database.

[1051] 3. The server preprocesses and tokenizes the text data.

[1052] 4. The generative model extracts keywords such as "recent movies" and "movies 2023" from the preprocessed data.

[1053] 5. The emotion engine recognizes User A's positive emotions and weights positive movie keywords.

[1054] 6. The server filters these keywords and removes inappropriate content.

[1055] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[1056] 8. When User A uses a search engine, suggestions such as "recent movies" and "movies 2023" appear.

[1057] Case 2: Talking about the weather

[1058] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[1059] 2. The server collects this message, encrypts it, and stores it in a database.

[1060] 3. The server preprocesses and tokenizes the text data.

[1061] 4. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast" from the preprocessed data.

[1062] 5. The emotion engine recognizes user C's anxious emotions and weights keywords for detailed weather information.

[1063] 6. The server filters these keywords and removes inappropriate content.

[1064] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[1065] 8. When User C uses a search engine, they will see emotion-based suggestions such as "tomorrow's weather" and "weather forecast."

[1066] An example of a prompt is:

[1067] "Are there any good movies out recently?"

[1068] "What's the weather going to be like tomorrow?"

[1069] In this way, the present invention can efficiently provide personalized search suggestions that take into account the user's communication data and emotions.

[1070] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1071] Step 1:

[1072] The server collects communication data from the user's device. Specifically, when a user chats using a messaging app, the text data is sent to the server via the network. At this time, the collected communication data is stored in a database using AES 256-bit encryption. The input is a text message from the user, and the output is encrypted communication data.

[1073] Step 2:

[1074] The server preprocesses the collected communication data. Specifically, it filters non-text information such as stamps, images, and videos from the text data, and divides the text into words using tokenization and morphological analysis tools (e.g., MeCab). The input to this step is the encrypted and stored communication data, and the output is clean text data after preprocessing.

[1075] Step 3:

[1076] The server inputs the preprocessed text data into a generative model. Specifically, it applies generative models such as BERT or GPT, which are implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model analyzes and extracts keywords that potentially interest the user from the preprocessed data. The input is the preprocessed text data, and the output is the extracted latent keywords.

[1077] Step 4:

[1078] The server simultaneously inputs the preprocessed text data into the emotion engine. The emotion engine uses natural language processing technology to recognize user emotions from the communication data. Specifically, it uses tools such as TextBlob and VADER to assign emotion labels such as positive, negative, and neutral to the data. The input is the preprocessed text data, and the output is the assigned emotion label.

[1079] Step 5:

[1080] The generative AI model weights the extracted keywords based on emotional data obtained from the emotion engine. For example, if a user expresses a positive emotion, the scoring algorithm is used to prioritize keywords that match that mood. The inputs are the extracted keywords and emotion labels, and the output is the weighted keywords.

[1081] Step 6:

[1082] The server filters the generated keywords to remove inappropriate content and spam. This process includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to ensure a good user experience. The input is weighted keywords, and the output is clean, filtered keywords.

[1083] Step 7:

[1084] The server finally integrates the filtered keywords into the search engine's suggestion function. Specifically, by uploading appropriate keywords as suggestion data using the search engine API, the suggestion function database is updated in real time. The input is the filtered keywords, and the output is the updated search engine's suggestion data.

[1085] Step 8:

[1086] When using a search engine, a user's device displays customized search suggestions, allowing the user to obtain search results optimized based on sentiment. For example, when searching for "recent movies," relevant suggestions such as "movies 2023" and "trending movies" are displayed. The input is a search query, and the output is search suggestions.

[1087] This allows users to experience personalized search suggestions based on emotions.

[1088] (Application example 2)

[1089] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1090] Conventional search engine suggestion functions are limited to suggesting keywords based on the user's communication data and do not take into account the user's emotional state, making it impossible to provide a truly personalized experience. In addition, user privacy may not be adequately protected, and ensuring safety is also an issue.

[1091] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into the search engine's suggestion function, means for recognizing the user's emotional state from the user's communication data, and means for weighting the generated search keywords based on the recognized emotional state. This makes it possible to suggest search keywords based on the user's emotional state, providing a more personalized user experience. Privacy protection is also ensured by anonymizing the communication data and emotional data.

[1092] "User Communication Data" means all data sent or received by a User through a Communication Device, including, in particular, text messages, voice data, and other information.

[1093] "Preprocessing" refers to a series of processes that remove unnecessary information from collected communication data and convert it into a format suitable for analysis.

[1094] A "generative model" refers to a machine learning algorithm that extracts and generates keywords that may be of interest to users based on collected and preprocessed data.

[1095] "Search engine suggestion function" refers to the function that predicts and displays related search keywords when a user types something into a search box.

[1096] "Means for recognizing emotional states" refers to algorithms that analyze a user's communication data and identify their emotional state (such as joy, sadness, or anxiety).

[1097] "Weighting" refers to the process of prioritizing generated keywords based on emotional state.

[1098] "Anonymization" refers to the process of removing personally identifiable information from collected data to protect user privacy.

[1099] The present invention relates to a system for generating search keywords based on communication data and emotional state of a user in a virtual store, and for making more personalized product suggestions. The system includes the following means.

[1100] A means of collecting and preprocessing user communication data

[1101] The server collects user communication data in real time or periodically via smartphones or head-mounted displays. The collected communication data is stored in an encrypted database (e.g., MySQL) on the server. Data security is ensured using encryption protocols such as TLS / SSL. Preprocessing involves filtering out unnecessary information (e.g., stamps, images, and videos) and converting text data into an appropriate format. This process makes the data easier to analyze.

[1102] A method using a generative model to generate potential search keywords from preprocessed communication data

[1103] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to extract search keywords that the user is likely to be interested in. This generative model can generate appropriate keywords based on past data and user behavior patterns.

[1104] A means to integrate generated search keywords into search engine suggestion functions

[1105] The extracted search keywords are filtered by the server to remove inappropriate content and spam, and then uploaded to the suggestion system (e.g., search engine API) within the virtual store and displayed when a user searches.

[1106] A means of recognizing a user's emotional state from their communication data

[1107] The server inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The emotional state is classified into major emotion categories such as joy, sadness, and anxiety. The recognition results play an important role in the subsequent keyword generation process.

[1108] A means of weighting generated search keywords based on perceived emotional states

[1109] The server adjusts the importance of generated search keywords based on the emotional data obtained from the emotion engine. For example, if a user feels like relaxing, keywords such as "relaxation goods" and "aroma" will be prioritized.

[1110] Specific examples

[1111] Specific scenarios

[1112] 1. User A searches for "relaxing products" in a virtual store.

[1113] 2. A smartphone app collects the user's relaxed emotional state along with their conversation data.

[1114] 3. The server preprocesses the data and inputs it into GPT-3 and the Microsoft Azure Emotion API.

[1115] 4. GPT-3 extracts keywords such as "relaxation," "relaxation goods," and "aroma."

[1116] 5. The Microsoft Azure Emotion API recognizes relaxed emotions.

[1117] 6. The suggestion system weights "relaxation," "relaxation goods," and "aroma" and integrates them into the virtual store's suggestion list.

[1118] 7. When User A browses the virtual store, products such as "relaxation goods" and "aroma" are displayed preferentially.

[1119] Prompt Sentence Examples

[1120] Collect chat data that shows users feeling relaxed and analyze it using an emotion engine. The generative AI model should extract keywords that users are likely to be interested in, such as "relaxation," "aroma," and "relaxation goods," and integrate these into the suggestion system.

[1121] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1122] Step 1:

[1123] The server collects the user's communication data from the smartphone or head-mounted display. The collected communication data is encrypted and stored in a database. The input is the user's communication data (text messages, voice data, and other information), and the output is the data stored in the database in encrypted form. This step includes specific operations to ensure the security of the data using the TLS / SSL protocol.

[1124] Step 2:

[1125] The server preprocesses the collected communication data. Specifically, it filters out unnecessary information (e.g., stamps, images, videos) and formats it as text data. The input is encrypted communication data, and the output is preprocessed text data. In this step, the data cleansing and filtering process is performed.

[1126] Step 3:

[1127] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to generate search keywords that the user is likely to be interested in. The input is the preprocessed text data, and the output is the generated search keywords. This step includes the specific operation of extracting keywords using a machine learning algorithm.

[1128] Step 4:

[1129] The server simultaneously inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The input is preprocessed text data, and the output is the recognized emotional state. In this step, natural language processing techniques are used to classify the user's emotion.

[1130] Step 5:

[1131] The server combines the search keywords obtained from the generative AI model with the emotional data obtained from the emotion engine and weights the keywords. The input is the generated search keywords and the recognized emotional state, and the output is the weighted search keywords. In this step, an algorithm is run to determine the priority of the keywords.

[1132] Step 6:

[1133] The server filters the weighted search keywords to remove inappropriate content and spam. The input is the weighted search keywords, and the output is the filtered keywords. This step includes specific operations to remove inappropriate keywords using text analysis techniques.

[1134] Step 7:

[1135] The server integrates the filtered keywords into the search engine's suggestion function. The input is the filtered keywords, and the output is the keywords uploaded as search engine suggestion data. In this step, the search engine API is used to add the keywords to the suggestion system.

[1136] Step 8:

[1137] When a user uses a virtual store, search suggestions customized based on their emotions are displayed. The input is the search engine's suggestion data, and the output is the customized suggestions displayed on the user's device. This step includes the specific operation of displaying appropriate suggestions through the user interface.

[1138] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1139] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1140] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1141] [Fourth embodiment]

[1142] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1143] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1144] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1145] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1146] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1147] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1148] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1149] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1150] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1151] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1152] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1153] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1154] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1155] This invention is a system that collects and analyzes user communication data and integrates it into the suggestion function of a search engine. This system can effectively suggest information that users have forgotten about or that they potentially want to know.

[1156] What the program does

[1157] 1. Collecting chat data

[1158] The server collects chat data from users' devices and stores it in an encrypted database. This collection process is performed periodically, allowing users to view their most recent chat data.

[1159] 2. Data Preprocessing

[1160] The server formats the collected chat data into a format suitable for the generative model. Specifically, it removes unnecessary information (e.g., stamps, images, and videos) from the text data and divides it into sentences. This process allows the generative model to effectively extract keywords.

[1161] 3. Data input to the generative model

[1162] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT and GPT. The model analyzes the chat data and extracts keywords that users are likely to be interested in.

[1163] 4. Keyword extraction and filtering

[1164] A generative model generates important keywords from the chat data, which are then further filtered to remove spam and inappropriate content, resulting in highly reliable keywords.

[1165] 5. Reflection in search suggestions

[1166] The server integrates the extracted keywords into the search engine's suggestion function, specifically by uploading the generated keywords as search suggestion data, which are displayed when a user starts typing in the search box.

[1167] 6. Displaying suggestions to users

[1168] When a user uses a search engine on their device, customized search suggestions are displayed, allowing the user to easily find relevant information.

[1169] Specific examples

[1170] Case 1: Movie talk

[1171] 1. User A and User B have a conversation on LINE about "Do you know what movies are out recently?"

[1172] 2. The server collects and pre-processes this data.

[1173] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[1174] 4. The server integrates these keywords into the search engine's suggestion function.

[1175] 5. When User A uses Yahoo Search, suggestions such as "Recent Movies" and "Movies 2023" are displayed, and the user can click on them to search for more information.

[1176] Case 2: Talking about the weather

[1177] 1. User C sends a message on LINE saying, "Do you know what the weather will be like tomorrow?"

[1178] 2. The server collects and preprocesses this message.

[1179] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[1180] 4. The server integrates these keywords into the search engine's suggestion function.

[1181] 5. When User C uses Yahoo Search, keywords such as "tomorrow's weather" and "weather forecast" are displayed in the suggestion box, allowing them to search directly.

[1182] Privacy and Data Security

[1183] To protect user privacy during data collection and analysis, the server anonymizes the collected data so that it cannot be used to identify individuals, and uses encryption protocols (e.g., TLS / SSL) for data transmission.

[1184] Users can opt out of this suggestion feature from the settings screen, providing peace of mind regarding their privacy.

[1185] In this way, the present invention provides a system that utilizes users' communication data to understand their latent search intent and reflect this in the search engine's suggestion function, thereby facilitating users' search activities.

[1186] The processing flow will be explained below.

[1187] Step 1:

[1188] The server collects chat data from user devices, retrieves it in real time or periodically via API, and stores it in a secure database. The data is protected using encryption technology while stored.

[1189] Step 2:

[1190] The server preprocesses the collected talk data into a format suitable for the generative model. Specifically, unnecessary information such as stamps, images, and videos are filtered from the text data, and the data is divided into talks and formatted as text tokens.

[1191] Step 3:

[1192] The server inputs the preprocessed chat data into a generative model, which uses natural language processing techniques such as BERT or GPT. This model analyzes the chat data and extracts search keywords that the user is likely to be interested in.

[1193] Step 4:

[1194] The generative model extracts keywords from the analysis of the chat data. Specifically, it selects and lists highly important keywords and phrases. These keywords then undergo a filtering process to remove inappropriate content and spam.

[1195] Step 5:

[1196] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, it uploads the generated keywords as suggestion data through the Yahoo! Search API. This upload process immediately updates the keywords to the suggestion list.

[1197] Step 6:

[1198] When a user uses a search engine on their device, the search engine displays customized search suggestions. As the user types in the search box, the keywords integrated in step 5 are displayed in real time.

[1199] Step 7:

[1200] Users can click on the suggested keywords and perform a search to retrieve related information, allowing them to quickly access the information they were potentially looking for.

[1201] Step 8:

[1202] The server collects data on users' search behavior and uses it to improve the system. Specifically, it collects data such as which suggestions users clicked and which search results they viewed, and reuses this data as training data for the generative model.

[1203] Step 9:

[1204] The generative model is retrained based on the collected feedback data, which improves the accuracy of keyword extraction the next time and allows the model to provide more appropriate suggestions to users.

[1205] Step 10:

[1206] Users can opt out of data collection and suggestion features in the settings screen, allowing them to use the service in a privacy-conscious manner.

[1207] Example 1

[1208] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1209] The suggestion functions of conventional search engines are based on users' past search history and general trends, and therefore often fail to effectively provide keywords that are relevant to the user's current interests and circumstances. Furthermore, from the perspective of privacy protection, there are sometimes concerns about whether user data is being managed appropriately. Therefore, there is a need for the development of a system that reflects the information users are seeking in everyday conversations in real time while protecting their privacy.

[1210] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1211] In this invention, the server includes means for collecting and preprocessing user communication data, means for removing unnecessary information from the preprocessed communication data and dividing it into sentence units, means for inputting the divided communication data into a generative model to generate potential search keywords, and means for filtering the generated search keywords and integrating them into the search engine's suggestion function. This makes it possible to provide a suggestion function that reflects the user's latest interests and situation, thereby achieving both an improved user experience and privacy protection.

[1212] "Communication data" refers to text information exchanged by users using chat applications, messaging services, etc.

[1213] "Preprocessing" is the process of removing unnecessary information from collected communication data and formatting it into a format that is easy to analyze.

[1214] A "generative model" is an algorithm or software that uses natural language processing techniques to generate potential search keywords from preprocessed communication data.

[1215] A "search engine suggestion function" is a search assistance function that automatically suggests related keywords when a user begins typing in the search box.

[1216] "Filtering" is the process of removing inappropriate content and spam from the generated keywords, leaving only reliable keywords.

[1217] An "encrypted database" is a database that is protected using encryption technology to ensure data security.

[1218] "Anonymization" is a process that protects privacy by removing personally identifiable information from collected communications data.

[1219] The present invention is a system that collects and preprocesses user communication data, and then integrates keywords generated from that data into the suggestion function of a search engine, thereby providing suggestion functions that reflect the user's latest interests and circumstances.

[1220] Hardware and software used

[1221] This system uses the following hardware and software:

[1222] Server: A central computing unit for data collection, preprocessing, running generative models, filtering keywords, and integrating them into suggestion functions. Specific examples include high-performance cloud servers (e.g., AWS, GCP).

[1223] User terminal: A device that generates communication data and displays suggestions. Examples include smartphones, tablets, and PCs.

[1224] Generative AI model: A generative model that uses natural language processing techniques. In particular, we use the latest generative AI models such as BERT and GPT.

[1225] Data processing and calculation

[1226] 1. Collecting Talk Data:

[1227] The server periodically collects chat data from the user's device. This process involves obtaining message data from chat apps such as the LINE application.

[1228] The collected data is stored in an encrypted database to ensure security.

[1229] 2. Data preprocessing:

[1230] The server retrieves the collected data from the encrypted database and removes unnecessary information (e.g. stamps, images, videos).

[1231] The preprocessed data is split into text sentences and converted into a clean format, removing special characters and unnecessary spaces in the process.

[1232] 3. Data input to the generative model:

[1233] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4), which generates potential search keywords from the user's conversation.

[1234] The generated keywords are output along with a confidence score.

[1235] 4. Keyword extraction and filtering:

[1236] The server runs the keywords obtained from the generative AI model through a filtering algorithm to remove inappropriate content and spam.

[1237] The filtered keywords are saved as a reliable list.

[1238] 5. Reflection in search suggestions:

[1239] The server uploads the filtered keywords to the search engine's suggestion database, which is updated in real time.

[1240] 6. Suggestions for users:

[1241] The user's device will display customized search suggestions through the browser or search application, allowing the user to efficiently access the information they need.

[1242] Specific examples

[1243] Some specific examples are given below.

[1244] Case 1: Movie talk

[1245] 1. User A and User B have a conversation about "Do you know what movies are out recently?"

[1246] 2. The server collects and preprocesses this talk data.

[1247] 3. The generative model generates keywords such as "recent movies" and "movies 2023."

[1248] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[1249] 5. When User A performs a web search, suggestions such as "recent movies" and "movies 2023" are displayed.

[1250] Case 2: Talking about the weather

[1251] 1. User C sends a message saying, "Do you know what the weather will be like tomorrow?"

[1252] 2. The server collects and preprocesses this message.

[1253] 3. The generative model generates keywords such as "tomorrow's weather" and "weather forecast."

[1254] 4. The server filters the keywords and integrates them into the search engine's suggestion function.

[1255] 5. When User C uses a search engine, keywords such as "tomorrow's weather" and "weather forecast" are suggested.

[1256] This mechanism makes it possible to provide effective suggestions based on the user's latest interests.

[1257] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1258] Step 1:

[1259] The server collects chat data from the user's device. Specifically, it obtains the message history of the chat application. For example, it collects message data from LINE such as "Do you know what movies are out these days?" The input is the user's communication data, and the output is the result of collecting this data. The data is encrypted and stored in a secure database.

[1260] Step 2:

[1261] The server preprocesses the collected chat data. The input is chat data read from an encrypted database, and the output is clean data with unnecessary information removed. Specifically, non-text information such as stamps, images, and videos in messages is removed, and the text is divided into sentences. Special characters and unnecessary spaces are also removed during this process.

[1262] Step 3:

[1263] The server inputs the preprocessed data into a generative model. The input is cleaned and formatted talk data, and the output is generated keywords and their confidence scores. Specifically, a generative AI model (e.g., GPT-4) is used to analyze the text and generate potential search keywords such as "recent movies" and "movies 2023."

[1264] Step 4:

[1265] The server inputs the keywords obtained from the generative model into a filtering algorithm. The input is the generated keywords, and the output is a filtered, more reliable list of keywords. Specific filtering actions include removing spam and inappropriate content. For example, if the keyword "recent movies" is appropriate, it will be left as is, but "spammy content" will be removed.

[1266] Step 5:

[1267] The server uploads the filtered keyword list to the search engine's suggestion database. The input is the filtered keyword list, and the output is a search suggestion function that integrates the list. Specific operations include writing to the database and updating it in real time. This process adds keywords such as "movies 2023" to the suggestion database.

[1268] Step 6:

[1269] When a user uses a search engine on their device, the generated keywords are displayed as suggestions in real time. The input is the user's search query, and the output is the keywords displayed in the suggestion box. Specifically, when a user begins to type in the search box, keywords such as "recent movies" and "weather forecast" are presented as candidates. As a result, users can efficiently access related information.

[1270] (Application example 1)

[1271] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1272] Conventional navigation systems are unable to understand the individual needs and preferences of users, and are limited to providing pre-set route information and destinations. This means that they are unable to appropriately suggest places and information that are truly of interest to users. In relation to this, there is a need for real-time, personalized navigation based on in-car conversations and behavior.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1274] In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative AI model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into a navigation system, and means for displaying the keywords generated based on conversation data collected in the car on the navigation system. This enables real-time keyword generation based on conversation data in the car and personalized destination and information suggestions based on the generated keywords.

[1275] "User communications data" is information generated by users, such as conversations in the car or text messages.

[1276] "Preprocessing" refers to the process of removing noise and unnecessary information from collected user communication data and shaping the data to make it easier to analyze using a generative model.

[1277] A "generative model" is a model that extracts latent keywords from user communication data that has been preprocessed using natural language processing technology. Specifically, AI models such as BERT and GPT fall into this category.

[1278] "Integrating into the navigation system" refers to the process of incorporating the keywords extracted by the generative model into the database of the vehicle's navigation system, enabling information to be provided in real time.

[1279] A "navigation system" is an electronic system installed in a vehicle that allows the user to set a destination and provides optimal route guidance.

[1280] "Conversational data collected in the vehicle" refers to the user's voice and text information obtained through the vehicle's microphone and text input device.

[1281] "Keywords" are important words or phrases extracted from communication data by the generative model and used in navigation systems to identify specific information or destinations.

[1282] "Displaying" refers to visually providing information and destinations related to the extracted keywords to the user on the display of the navigation system.

[1283] A system for implementing this invention is a navigation system installed in an autonomous vehicle that generates potential keywords based on user communication data and uses them to provide personalized navigation guidance.

[1284] System Configuration

[1285] The system includes the following major components:

[1286] 1. In-vehicle server

[1287] 2. Generative Model

[1288] 3. Navigation system

[1289] 4. In-car displays

[1290] 5. Microphones and Text Input Devices

[1291] What the program does

[1292] In-vehicle server

[1293] The in-vehicle server collects conversation data through the vehicle's microphone and text input devices. The collected data is stored in an encrypted database. This collection process is performed periodically to obtain the latest data.

[1294] Pretreatment

[1295] The in-vehicle server preprocesses the collected conversation data into a format that is easy to analyze. Specifically, it removes noise and unnecessary information and divides the data into sentences. This preprocessing allows the generative model to effectively extract keywords.

[1296] Generative Model

[1297] The preprocessed conversation data is input to a generative model (e.g., BERT or GPT), which analyzes the data and extracts keywords that the user is likely to be interested in from the conversations in the car.

[1298] Integration into navigation systems

[1299] The in-vehicle server integrates the generated keywords into the navigation system, which then uses these customized keywords to display personalized routes and information when the user sets a destination.

[1300] Displaying suggestions to users

[1301] The in-car display will show customized suggestions through the navigation system, allowing users to easily find relevant information and enjoy greater convenience.

[1302] Hardware and software used

[1303] The system includes the following specific hardware and software:

[1304] In-vehicle server: A server installed in the vehicle for collecting and analyzing data

[1305] Microphone and text input device: A device for collecting voice data and text information inside the vehicle.

[1306] Generative models: AI models that use natural language processing techniques such as BERT and GPT

[1307] Navigation system: In-vehicle route guidance and information display system

[1308] Encryption protocol: Technology to ensure security of data communication such as TLS / SSL

[1309] Examples and prompts

[1310] Specific examples

[1311] Case 1: A user asks in the car, "Is there a good ramen restaurant nearby?"

[1312] An in-vehicle server collects and pre-processes this data.

[1313] GPT-4 extracts keywords such as "delicious ramen restaurant" and "recommended ramen restaurant."

[1314] The extracted keywords are integrated into the navigation system.

[1315] When the user launches the navigation system, suggestions such as "delicious ramen restaurants" are displayed, and navigation can be started immediately.

[1316] Prompt Sentence Examples

[1317] Generate keywords that will attract users' interest based on the latest in-car conversation data. Example keywords are shown below.

[1318] "Delicious ramen restaurant"

[1319] "Nearby gas station"

[1320] "Playground for children"

[1321] In this way, the system of the present invention can improve the convenience and personalized experience of autonomous vehicles by utilizing the user's real-time communication data and enhancing the navigation system's suggestion functions.

[1322] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1323] Step 1:

[1324] The server collects the user's communication data (conversation data and text messages) from the microphone and text input device. The input is the user's real-time conversation data, and the output is formatted data stored in an encrypted database. This collection process is performed periodically to capture the latest communication data occurring inside the vehicle.

[1325] Step 2:

[1326] The server preprocesses the collected conversation data. Specifically, it removes noise and unnecessary information and divides it into sentences. The input is raw data from an encrypted database, and the output is noise-removed, formatted text data. This preprocessing allows the generative model to effectively extract keywords.

[1327] Step 3:

[1328] The server inputs the preprocessed conversation data into a generative model (e.g., GPT-4 or BERT). The generative model analyzes the conversation data and extracts keywords of potential interest. The input is preprocessed text data, and the output is a list of keywords as the analysis result.

[1329] Step 4:

[1330] The server integrates the generated keywords into the navigation system. Specifically, it adds the generated keywords to the navigation system database or displays them as real-time search suggestions. The input is the generated keyword list, and the output is the suggestion data integrated into the navigation system.

[1331] Step 5:

[1332] The navigation system will suggest destinations and related information to the user based on the generated keywords through the in-car display. The input is the suggestion data integrated into the navigation system, and the output is customized destination and route information displayed on the display. This step allows the user to intuitively browse customized destination information.

[1333] Step 6:

[1334] When the user selects a suggested keyword on the in-car display, the navigation system generates the optimal route based on the selected destination and information and begins providing guidance. The input is the user's selected keyword, and the output is the specific guided route.

[1335] This series of processing steps allows users to enjoy personalized navigation based on the conversations they have in the car, greatly improving convenience.

[1336] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1337] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines an emotion engine that recognizes the user's emotions to provide more appropriate search suggestions based on the user's emotional state. This makes it possible to further personalize the user experience and improve satisfaction.

[1338] What the program does

[1339] 1. Collecting chat data

[1340] The server collects chat data from user devices and stores it in encrypted form in a secure database. This collection can occur in real time or periodically.

[1341] 2. Data Preprocessing

[1342] The server preprocesses the collected chat data, specifically filtering out unnecessary information (e.g., stickers, images, videos) and formatting it into a format suitable for the generative model.

[1343] 3. Data input to the generative model

[1344] The server inputs the preprocessed chat data into a generative model (e.g., BERT or GPT), which analyzes this data and extracts keywords that potentially interest the user.

[1345] 4. Introducing the Emotion Engine

[1346] The server inputs the same data into the emotion engine in parallel with the conversation data. The emotion engine uses natural language processing technology to recognize emotions from the user's conversation.

[1347] 5. Keyword extraction and sentiment consideration

[1348] The generative AI model analyzes the chat data to generate important keywords, and the emotion engine weights the keywords based on the user's emotional data. For example, if the user is expressing positive emotions, it will prioritize keywords that match that mood.

[1349] 6. Keyword Filtering

[1350] The server filters the generated keywords to remove inappropriate content and spam, a process that also takes sentiment data into account.

[1351] 7. Reflection in suggestion function

[1352] The server then integrates the extracted keywords into the search engine's suggestion function, taking into account emotional data. Specifically, it uses the Yahoo! Search API to upload appropriate keywords as suggestion data.

[1353] 8. Displaying suggestions to users

[1354] When a user uses a search engine on their device, they will see personalized search suggestions based on their emotions, helping them quickly find the information that best matches their search intent.

[1355] Specific examples

[1356] Case 1: Movie talk

[1357] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[1358] 2. The server collects and preprocesses this talk data.

[1359] 3. The generative model extracts keywords such as "recent movies" and "movies 2023."

[1360] 4. The emotion engine recognizes user A's positive emotions and weights positive movie keywords that match those emotions.

[1361] 5. The server integrates these keywords into the search engine's suggestion function, and when User A uses Yahoo Search, emotion-based suggestions such as "recent movies" and "movies 2023" are displayed.

[1362] Case 2: Talking about the weather

[1363] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[1364] 2. The server collects and preprocesses this message.

[1365] 3. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast."

[1366] 4. The emotion engine recognizes user C's anxious emotion and weights keywords for detailed weather information that correspond to that emotion.

[1367] 5. The server integrates these keywords into the search engine's suggestion function, and when User C uses Yahoo Search, detailed suggestions such as "tomorrow's weather" and "weather forecast" based on emotions are displayed.

[1368] Privacy and Data Security

[1369] During the data collection and analysis process, the server anonymizes the collected data to protect user privacy and makes it impossible to identify individuals. In addition, encryption protocols (e.g., TLS / SSL) are used for data transmission.

[1370] Users can opt out of data collection, sentiment analysis, and suggestion functions from the settings screen, allowing them to use the service in a privacy-conscious manner.

[1371] The present invention significantly improves user experience by providing search suggestions that take into account the user's communication data and their sentiments.

[1372] The processing flow will be explained below.

[1373] Step 1:

[1374] The server collects chat data from user devices, periodically retrieves the data via API, and stores it in a secure database. The data is protected using AES encryption technology when stored.

[1375] Step 2:

[1376] The server preprocesses the collected chat data. Specifically, it filters unnecessary information (e.g., stamps, images, videos) from the text data, divides it into chats, and formats it appropriately.

[1377] Step 3:

[1378] The server inputs the preprocessed talk data into a generative model (e.g., a natural language processing model such as BERT or GPT), which analyzes the talk data and extracts important keywords.

[1379] Step 4:

[1380] The server also inputs the chat data into the emotion engine, which uses natural language processing technology to analyze and recognize emotions from the user's text. The analysis results are output as emotion labels such as anger, joy, and sadness.

[1381] Step 5:

[1382] The generative model generates important keywords based on the preprocessed chat data. At this time, the emotion engine references the emotional data recognized and weights the keywords. For example, if the user is expressing positive emotions, it will prioritize keywords related to happy content.

[1383] Step 6:

[1384] The server-generated keywords are filtered to remove inappropriate content and spam from the keyword list and select only reliable keywords. Sentiment data is also taken into account in this filtering process.

[1385] Step 7:

[1386] The server integrates the extracted keywords into the search engine's suggestion function. Specifically, the keywords are uploaded as suggestion data via the search engine's API. This allows emotion-sensitive keywords to be displayed as suggestions in the search box.

[1387] Step 8:

[1388] When a user uses a search engine on their device, search suggestions customized based on their emotions are displayed. When a user types text into the search box, keywords integrated by the server are displayed as suggestions in real time.

[1389] Step 9:

[1390] Users can click on suggested keywords and perform a search to quickly retrieve relevant information, which is a process that best matches the user's search intent and increases user satisfaction.

[1391] Step 10:

[1392] The server collects data on users' search behavior and uses it to improve the system, such as which suggestions were clicked and which search results were viewed, and uses this data to train the generative model and emotion engine.

[1393] Step 11:

[1394] The generative model is retrained based on the collected feedback data to improve its keyword extraction accuracy the next time, which allows it to provide more relevant suggestions to users.

[1395] Step 12:

[1396] Users can opt out of data collection, sentiment analysis, and suggestion functions through the settings screen. This protects users' privacy and provides an environment where users can use the service with peace of mind.

[1397] Example 2

[1398] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1399] The suggestion function of conventional search engines presented keywords without taking into account the user's emotions or the content of their communication data, which meant that they were unable to fully meet the needs of users. In addition, they lacked privacy protection, making it difficult to provide an environment where users could use the search engine with peace of mind.

[1400] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for weighting the generated search keywords based on the user's emotion data, means for filtering the generated keywords to remove inappropriate content and spam, and means for integrating the generated search keywords into the suggestion function of the search engine. This makes it possible to provide personalized search suggestions that reflect the user's emotions and the content of the communication data, while also ensuring privacy protection.

[1401] "Communications Data" refers to information such as text messages, audio data, and video data that a user sends or receives.

[1402] "Preprocessing" is the process of removing unnecessary information from collected communication data and converting it into a format suitable for analysis and processing.

[1403] A "generative model" is an algorithm or machine learning model that uses natural language processing technology to extract potential search keywords from preprocessed communication data.

[1404] "Emotion data" is data that indicates a user's emotions and psychological state, recognized from communication data using natural language processing technology.

[1405] "Weighting" is the process of assigning a certain score or priority to the generated keywords to adjust their importance.

[1406] "Filtering" is the process of removing inappropriate content and spam from the generated keywords.

[1407] A "suggestion function" is a function that automatically suggests related keywords in response to a query entered by a user in a search engine.

[1408] "Privacy protection" refers to the means and technologies that anonymize users' personal information and communication data and protect them from leaks to third parties.

[1409] "Encryption" is the technology of transforming data using specific algorithms so that only authorized persons can decipher it.

[1410] "Data collection" is the process of importing user communication data via a network into a server or database.

[1411] "Natural language processing" is the field of technology that enables computers to understand, generate, and manipulate human language.

[1412] This invention is a system that collects user communication data, analyzes that data, generates search keywords, and integrates them into the suggestion function of a search engine. It also combines this with an emotion engine that recognizes the user's emotions, to provide more appropriate search suggestions based on the user's emotional state.

[1413] The server first collects user communication data. This collected data is then stored in a secure database in encrypted form using AES 256-bit encryption. The collected data is then sent to the server in real time or periodically.

[1414] The server then pre-processes the collected communication data. This involves filtering out media data such as stamps, images, and videos from the text. This data is then tokenized using a morphological analysis tool such as MeCab, and converted into a format suitable for analysis and processing.

[1415] The preprocessed data is then input into a generative model, such as BERT or GPT, which is implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model understands the context of the chat data and extracts keywords that potentially interest users.

[1416] In parallel, the server also inputs the same chat data into the emotion engine, which uses natural language processing (NLP) to recognize the user's emotions from the communication data. Emotions are classified into labels such as positive, negative, and neutral. Tools such as TextBlob and VADER are used for this emotion analysis.

[1417] The keywords extracted by the generative AI model are then weighted based on the emotional data recognized by the emotion engine. The weighting is done using a scoring algorithm to prioritize keywords that match a user's positive emotion.

[1418] The generated keywords then go through a filtering process, which removes inappropriate content and spam. This includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to remove inappropriate keywords, improving the quality of the user experience.

[1419] The filtered keywords are finally integrated into the search engine's suggestion function. Appropriate keywords are uploaded as suggestion data using the Yahoo! Search API, etc. This updates the suggestion database in real time.

[1420] When users use search engines, they will see personalized search suggestions, which will give users search suggestions optimized based on sentiment. For example, if you search for "recent movies," you will see relevant suggestions such as "movies 2023" and "trending movies."

[1421] Specific examples include the following cases:

[1422] Case 1: Movie talk

[1423] 1. User A and User B have a conversation about "Are there any good movies out recently?"

[1424] 2. The server collects this chat data, encrypts it, and stores it in a database.

[1425] 3. The server preprocesses and tokenizes the text data.

[1426] 4. The generative model extracts keywords such as "recent movies" and "movies 2023" from the preprocessed data.

[1427] 5. The emotion engine recognizes User A's positive emotions and weights positive movie keywords.

[1428] 6. The server filters these keywords and removes inappropriate content.

[1429] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[1430] 8. When User A uses a search engine, suggestions such as "recent movies" and "movies 2023" appear.

[1431] Case 2: Talking about the weather

[1432] 1. User C sends a message saying, "What's the weather going to be like tomorrow?"

[1433] 2. The server collects this message, encrypts it, and stores it in a database.

[1434] 3. The server preprocesses and tokenizes the text data.

[1435] 4. The generative model extracts keywords such as "tomorrow's weather" and "weather forecast" from the preprocessed data.

[1436] 5. The emotion engine recognizes user C's anxious emotions and weights keywords for detailed weather information.

[1437] 6. The server filters these keywords and removes inappropriate content.

[1438] 7. The server integrates the extracted keywords into the search engine's suggestion function.

[1439] 8. When User C uses a search engine, they will see emotion-based suggestions such as "tomorrow's weather" and "weather forecast."

[1440] An example of a prompt is:

[1441] "Are there any good movies out recently?"

[1442] "What's the weather going to be like tomorrow?"

[1443] In this way, the present invention can efficiently provide personalized search suggestions that take into account the user's communication data and emotions.

[1444] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1445] Step 1:

[1446] The server collects communication data from the user's device. Specifically, when a user chats using a messaging app, the text data is sent to the server via the network. At this time, the collected communication data is stored in a database using AES 256-bit encryption. The input is a text message from the user, and the output is encrypted communication data.

[1447] Step 2:

[1448] The server preprocesses the collected communication data. Specifically, it filters non-text information such as stamps, images, and videos from the text data, and divides the text into words using tokenization and morphological analysis tools (e.g., MeCab). The input to this step is the encrypted and stored communication data, and the output is clean text data after preprocessing.

[1449] Step 3:

[1450] The server inputs the preprocessed text data into a generative model. Specifically, it applies generative models such as BERT or GPT, which are implemented using deep learning frameworks such as TensorFlow or PyTorch. The generative model analyzes and extracts keywords that potentially interest the user from the preprocessed data. The input is the preprocessed text data, and the output is the extracted latent keywords.

[1451] Step 4:

[1452] The server simultaneously inputs the preprocessed text data into the emotion engine. The emotion engine uses natural language processing technology to recognize user emotions from the communication data. Specifically, it uses tools such as TextBlob and VADER to assign emotion labels such as positive, negative, and neutral to the data. The input is the preprocessed text data, and the output is the assigned emotion label.

[1453] Step 5:

[1454] The generative AI model weights the extracted keywords based on emotional data obtained from the emotion engine. For example, if a user expresses a positive emotion, the scoring algorithm is used to prioritize keywords that match that mood. The inputs are the extracted keywords and emotion labels, and the output is the weighted keywords.

[1455] Step 6:

[1456] The server filters the generated keywords to remove inappropriate content and spam. This process includes blacklist-based filtering and quality assessment using language models. Sentiment data is also taken into account to ensure a good user experience. The input is weighted keywords, and the output is clean, filtered keywords.

[1457] Step 7:

[1458] The server finally integrates the filtered keywords into the search engine's suggestion function. Specifically, by uploading appropriate keywords as suggestion data using the search engine API, the suggestion function database is updated in real time. The input is the filtered keywords, and the output is the updated search engine's suggestion data.

[1459] Step 8:

[1460] When using a search engine, a user's device displays customized search suggestions, allowing the user to obtain search results optimized based on sentiment. For example, when searching for "recent movies," relevant suggestions such as "movies 2023" and "trending movies" are displayed. The input is a search query, and the output is search suggestions.

[1461] This allows users to experience personalized search suggestions based on emotions.

[1462] (Application example 2)

[1463] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1464] Conventional search engine suggestion functions are limited to suggesting keywords based on the user's communication data and do not take into account the user's emotional state, making it impossible to provide a truly personalized experience. In addition, user privacy may not be adequately protected, and ensuring safety is also an issue.

[1465] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and preprocessing user communication data, means for using a generative model to generate potential search keywords from the preprocessed communication data, means for integrating the generated search keywords into the search engine's suggestion function, means for recognizing the user's emotional state from the user's communication data, and means for weighting the generated search keywords based on the recognized emotional state. This makes it possible to suggest search keywords based on the user's emotional state, providing a more personalized user experience. Privacy protection is also ensured by anonymizing the communication data and emotional data.

[1466] "User Communication Data" means all data sent or received by a User through a Communication Device, including, in particular, text messages, voice data, and other information.

[1467] "Preprocessing" refers to a series of processes that remove unnecessary information from collected communication data and convert it into a format suitable for analysis.

[1468] A "generative model" refers to a machine learning algorithm that extracts and generates keywords that may be of interest to users based on collected and preprocessed data.

[1469] "Search engine suggestion function" refers to the function that predicts and displays related search keywords when a user types something into a search box.

[1470] "Means for recognizing emotional states" refers to algorithms that analyze a user's communication data and identify their emotional state (such as joy, sadness, or anxiety).

[1471] "Weighting" refers to the process of prioritizing generated keywords based on emotional state.

[1472] "Anonymization" refers to the process of removing personally identifiable information from collected data to protect user privacy.

[1473] The present invention relates to a system for generating search keywords based on communication data and emotional state of a user in a virtual store, and for making more personalized product suggestions. The system includes the following means.

[1474] A means of collecting and preprocessing user communication data

[1475] The server collects user communication data in real time or periodically via smartphones or head-mounted displays. The collected communication data is stored in an encrypted database (e.g., MySQL) on the server. Data security is ensured using encryption protocols such as TLS / SSL. Preprocessing involves filtering out unnecessary information (e.g., stamps, images, and videos) and converting text data into an appropriate format. This process makes the data easier to analyze.

[1476] A method using a generative model to generate potential search keywords from preprocessed communication data

[1477] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to extract search keywords that the user is likely to be interested in. This generative model can generate appropriate keywords based on past data and user behavior patterns.

[1478] A means to integrate generated search keywords into search engine suggestion functions

[1479] The extracted search keywords are filtered by the server to remove inappropriate content and spam, and then uploaded to the suggestion system (e.g., search engine API) within the virtual store and displayed when a user searches.

[1480] A means of recognizing a user's emotional state from their communication data

[1481] The server inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The emotional state is classified into major emotion categories such as joy, sadness, and anxiety. The recognition results play an important role in the subsequent keyword generation process.

[1482] A means of weighting generated search keywords based on perceived emotional states

[1483] The server adjusts the importance of generated search keywords based on the emotional data obtained from the emotion engine. For example, if a user feels like relaxing, keywords such as "relaxation goods" and "aroma" will be prioritized.

[1484] Specific examples

[1485] Specific scenarios

[1486] 1. User A searches for "relaxing products" in a virtual store.

[1487] 2. A smartphone app collects the user's relaxed emotional state along with their conversation data.

[1488] 3. The server preprocesses the data and inputs it into GPT-3 and the Microsoft Azure Emotion API.

[1489] 4. GPT-3 extracts keywords such as "relaxation," "relaxation goods," and "aroma."

[1490] 5. The Microsoft Azure Emotion API recognizes relaxed emotions.

[1491] 6. The suggestion system weights "relaxation," "relaxation goods," and "aroma" and integrates them into the virtual store's suggestion list.

[1492] 7. When User A browses the virtual store, products such as "relaxation goods" and "aroma" are displayed preferentially.

[1493] Prompt Sentence Examples

[1494] Collect chat data that shows users feeling relaxed and analyze it using an emotion engine. The generative AI model should extract keywords that users are likely to be interested in, such as "relaxation," "aroma," and "relaxation goods," and integrate these into the suggestion system.

[1495] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1496] Step 1:

[1497] The server collects the user's communication data from the smartphone or head-mounted display. The collected communication data is encrypted and stored in a database. The input is the user's communication data (text messages, voice data, and other information), and the output is the data stored in the database in encrypted form. This step includes specific operations to ensure the security of the data using the TLS / SSL protocol.

[1498] Step 2:

[1499] The server preprocesses the collected communication data. Specifically, it filters out unnecessary information (e.g., stamps, images, videos) and formats it as text data. The input is encrypted communication data, and the output is preprocessed text data. In this step, the data cleansing and filtering process is performed.

[1500] Step 3:

[1501] The server inputs the preprocessed communication data into a generative AI model (e.g., GPT-3) to generate search keywords that the user is likely to be interested in. The input is the preprocessed text data, and the output is the generated search keywords. This step includes the specific operation of extracting keywords using a machine learning algorithm.

[1502] Step 4:

[1503] The server simultaneously inputs the communication data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. The input is preprocessed text data, and the output is the recognized emotional state. In this step, natural language processing techniques are used to classify the user's emotion.

[1504] Step 5:

[1505] The server combines the search keywords obtained from the generative AI model with the emotional data obtained from the emotion engine and weights the keywords. The input is the generated search keywords and the recognized emotional state, and the output is the weighted search keywords. In this step, an algorithm is run to determine the priority of the keywords.

[1506] Step 6:

[1507] The server filters the weighted search keywords to remove inappropriate content and spam. The input is the weighted search keywords, and the output is the filtered keywords. This step includes specific operations to remove inappropriate keywords using text analysis techniques.

[1508] Step 7:

[1509] The server integrates the filtered keywords into the search engine's suggestion function. The input is the filtered keywords, and the output is the keywords uploaded as search engine suggestion data. In this step, the search engine API is used to add the keywords to the suggestion system.

[1510] Step 8:

[1511] When a user uses a virtual store, search suggestions customized based on their emotions are displayed. The input is the search engine's suggestion data, and the output is the customized suggestions displayed on the user's device. This step includes the specific operation of displaying appropriate suggestions through the user interface.

[1512] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1513] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1514] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1515] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1516] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1517] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1518] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1519] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, motorcycles, and other devices, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1520] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1521] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1522] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1523] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1524] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1525] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1526] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1527] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1528] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1529] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1530] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1531] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1532] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1533] The following is further disclosed regarding the above embodiment.

[1534] (Claim 1)

[1535] means for collecting and preprocessing user communication data;

[1536] A means for using a generative model to generate potential search keywords from preprocessed communication data;

[1537] A means for integrating the generated search keywords into a search engine suggestion function;

[1538] A system including:

[1539] (Claim 2)

[1540] 2. The system according to claim 1, wherein keywords generated from the user's communication data are displayed in the search engine's suggestion function.

[1541] (Claim 3)

[1542] 10. The system of claim 1, further comprising a means for anonymizing user communication data to ensure privacy protection.

[1543] "Example 1"

[1544] (Claim 1)

[1545] means for collecting and preprocessing user communication data;

[1546] means for removing unnecessary information from the preprocessed communication data and dividing the data into sentence units;

[1547] A means for inputting the segmented communication data into a generative model to generate potential search keywords;

[1548] A means for filtering and integrating generated search keywords into search engine suggestion functions;

[1549] A system including:

[1550] (Claim 2)

[1551] 2. The system according to claim 1, wherein keywords generated from the user's communication data are displayed in the search engine's suggestion function.

[1552] (Claim 3)

[1553] 10. The system of claim 1, further comprising a means for anonymizing user communication data to ensure privacy protection.

[1554] "Application Example 1"

[1555] (Claim 1)

[1556] means for collecting and preprocessing user communication data;

[1557] A means for using a generative model to generate potential search keywords from preprocessed communication data;

[1558] means for integrating the generated search keywords into a navigation system;

[1559] a means for displaying keywords generated based on conversation data collected in the vehicle on a navigation system;

[1560] A system including:

[1561] (Claim 2)

[1562] 10. The system of claim 1, wherein the keywords generated from the user's communication data are displayed in a navigation system.

[1563] (Claim 3)

[1564] 10. The system of claim 1, further comprising a means for anonymizing user communication data to ensure privacy protection.

[1565] "Example 2: Combining Emotion Engines"

[1566] (Claim 1)

[1567] means for collecting and preprocessing user communication data;

[1568] A means for using a generative model to generate potential search keywords from preprocessed communication data;

[1569] A means for weighting the generated search keywords based on user sentiment data;

[1570] A means of filtering the generated keywords to remove inappropriate content and spam;

[1571] A means for integrating the generated search keywords into a search engine suggestion function;

[1572] A system including:

[1573] (Claim 2)

[1574] 2. The system according to claim 1, wherein keywords generated from the user's communication data are displayed in the search engine's suggestion function.

[1575] (Claim 3)

[1576] 10. The system of claim 1, further comprising a means for anonymizing user communication data to ensure privacy protection.

[1577] "Application example 2 when combining emotion engines"

[1578] (Claim 1)

[1579] means for collecting and preprocessing user communication data;

[1580] A means for using a generative model to generate potential search keywords from preprocessed communication data;

[1581] A means for integrating the generated search keywords into a search engine suggestion function;

[1582] means for recognizing an emotional state of a user from communication data;

[1583] means for weighting the generated search keywords based on the recognized emotional state;

[1584] A system including:

[1585] (Claim 2)

[1586] 10. The system of claim 1, wherein the search engine suggestion function displays keywords generated based on the user's communication data and emotional state.

[1587] (Claim 3)

[1588] 10. The system of claim 1, further comprising means for anonymizing the user's communication data and emotional state to ensure privacy. [Explanation of symbols]

[1589] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting and preprocessing user communication data; A means for using a generative model to generate potential search keywords from preprocessed communication data; A means for integrating the generated search keywords into a search engine suggestion function; A system including:

2. 2. The system according to claim 1, wherein keywords generated from the user's communication data are displayed in the search engine's suggestion function.

3. 10. The system of claim 1, further comprising a means for anonymizing user communication data to ensure privacy protection.

Images