system

Abstract

Provide a system. [Solution] means for analyzing the collected data using natural language analysis techniques; a means for removing noise from the collected data; means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms; means for collecting data based on user-specified keywords; means for displaying the generated summary on a user's terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's society, it is extremely difficult to keep track of all relevant news and social media posts. Furthermore, specialized skills are required to extract the necessary data from the vast amount of information, analyze it, and incorporate it into strategies. Trends and buzzwords change constantly, so it's essential to stay up-to-date with the latest information. In these circumstances, there is a need to perform dynamic data analysis in real time and present trend information in an easy-to-understand format for users. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. Specifically, by providing a means for analyzing collected data using natural language analysis technology, massive amounts of text data can be effectively analyzed and relevant information can be extracted. Next, by providing a means for removing noise from the collected data, the accuracy of the analysis can be improved. Furthermore, by providing a means for extracting related trends and buzzwords using topic modeling and clustering algorithms, important topics can be appropriately identified. Furthermore, by providing a means for collecting data based on keywords specified by the user, information that matches the user's interests can be collected. Finally, by providing a means for displaying the generated summary on the user's device, user convenience is improved. In this way, the system of the present invention makes it easier for users to understand trends and buzzwords in real time.

[0006] "Natural language analysis technology" refers to technology used to understand text data and extract meaning, and includes techniques such as morphological analysis and topic modeling.

[0007] "Topic modeling" is a technique for automatically extracting topics latent in large amounts of document data, and one example is the latent Dirichlet allocation method.

[0008] A "clustering algorithm" is an algorithm for grouping data based on similarity, such as the k-means algorithm.

[0009] "Noise reduction" refers to the process of removing unnecessary information (e.g., advertisements and spam) to improve the accuracy of data analysis.

[0010] "Collection means" refers to the methods and technologies used to acquire data; a typical example is data collection using an API.

[0011] "Analysis tools" refers to techniques and methods for processing collected data and extracting meaningful information.

[0012] "Trend extraction" refers to the process of identifying frequently occurring keywords and topics and listing them.

[0013] "Summary generation" refers to the process of concisely summarizing key information based on the analysis results and presenting it in a form that is easy for the user to understand.

[0014] "User terminal" refers to a device that is directly operated by a user, including, for example, a personal computer or smartphone. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a data analysis system that can grasp trends and buzzwords in real time, and performs automatic data collection and analysis to extract information from news and social media. Each element that makes up this system is explained below.

[0037] Data collection

[0038] The server collects data using APIs from multiple social media platforms and news sites based on keywords entered by the user on the device. For example, it uses the Twitter API to obtain the latest tweets about "AI technology." It also uses the Google (registered trademark) News API to collect related news articles. The collected data is stored in the server's database.

[0039] Data Preprocessing

[0040] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, etc., and removes unnecessary duplicate data.

[0041] Natural language analysis (NLP) technology

[0042] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation - LDA) is then used to identify major topics. After this, a clustering algorithm (e.g., k-means clustering) is used to group related topics.

[0043] Trend and buzzword extraction

[0044] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[0045] Summary Generation

[0046] Based on the extracted trends and buzzwords, the server generates an easy-to-understand summary for the user. For example, a summary such as "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 (registered trademark) and BERT are popular." This summary is formatted in HTML.

[0047] View Summary

[0048] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[0049] Specific examples

[0050] When a user types "latest AI technology trends" into their device, the server collects related data using the Twitter API and Google News API. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. A summary generated based on the extraction results is sent to the user's device and displayed. The user can view this information in real time and continue the analysis by entering new topics and keywords.

[0051] In this way, the system of the present invention utilizes natural language analysis technology to extract, analyze, and display important trends and buzzwords from vast amounts of data in real time, thereby supporting users' knowledge gathering and decision-making.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] User enters topic or keyword

[0055] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[0056] Step 2:

[0057] Server collects data from social media and news sites

[0058] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and stores the collected data in a local database.

[0059] Step 3:

[0060] The server preprocesses the data

[0061] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[0062] Step 4:

[0063] The server performs natural language processing (NLP)

[0064] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[0065] Step 5:

[0066] Trend and buzzword extraction

[0067] The server identifies frequently occurring keywords and phrases from the NLP analysis results, and lists major trends and buzzwords based on the results of topic modeling and clustering.

[0068] Step 6:

[0069] The server generates the summary

[0070] The server generates a summary for the user based on the extracted trends and buzzwords. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular."

[0071] Step 7:

[0072] The server sends the summary to the device

[0073] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[0074] Step 8:

[0075] The device displays a summary to the user

[0076] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[0077] Example 1

[0078] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0079] It solves the difficulty of quickly and effectively identifying trends and buzzwords from vast amounts of data and providing information that interests users in real time.

[0080] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0081] In this invention, the server includes means for collecting data based on keywords specified by a user, means for removing noise from the collected data, means for preprocessing the collected data, means for analyzing the preprocessed data using natural language analysis techniques, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for generating summaries based on the extracted trends and buzzwords, and means for displaying the generated summaries on the user's terminal, thereby enabling the latest information of interest to the user to be quickly and efficiently identified and displayed.

[0082] "Means for collecting data based on user-specified keywords" is a function that allows a user to input a specific keyword of interest and obtain information related to that keyword from multiple sources.

[0083] "Means for removing noise from collected data" refers to a processing function for removing unnecessary information, advertisements, and spam messages from collected data and improving the quality of the data.

[0084] "Means for preprocessing collected data" refers to a function for formatting collected data into a format that is easy to analyze, such as converting uppercase letters to lowercase, removing special characters, and deleting unnecessary duplicate data.

[0085] "Means for analyzing preprocessed data using natural language processing technology" refers to a function that utilizes natural language processing technology to further analyze preprocessed data and divide it into meaningful units. Specifically, this includes morphological analysis and topic modeling.

[0086] "Means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms" refers to the function of identifying important topics and frequently occurring keywords and phrases from collected and preprocessed data and grouping them. Latent Dirichlet Allocation is used for topic modeling, and the k-means algorithm is used for clustering.

[0087] "Means for generating summaries based on extracted trends and buzzwords" is a function for generating easy-to-understand summaries for users based on important information identified from the analyzed data.

[0088] The "means for displaying the generated summary on the user's terminal" is a function for sending the summary information generated by the server to the user's terminal and displaying it to the user via a browser or a dedicated application.

[0089] MODE FOR CARRYING OUT THE INVENTION

[0090] This invention relates to a data analysis system that extracts and displays in real time the latest trends and buzzwords that users are interested in. This system has the ability to automatically collect and analyze data and obtain information related to specific keywords from multiple sources, such as social media and news sites.

[0091] Data collection

[0092] The server collects data using APIs of social media and news sites based on keywords entered by the user on the device. For example, the Twitter API or Google News API can be used. A specific example is collecting the latest tweets and news articles related to "AI technology." The collected data is stored in the server's database.

[0093] Data Preprocessing

[0094] The collected data is pre-processed by the server. Filtering algorithms are used to remove advertisements and spam, and text data is normalized. This includes converting uppercase to lowercase, removing special characters, and deleting unnecessary duplicate data. The pre-processed data is then stored in a clean database.

[0095] Natural language analysis (NLP) technology

[0096] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Morphological analysis is performed to tokenize the text. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. Furthermore, the k-means clustering algorithm is used to group related topics to extract important trends and buzzwords.

[0097] Summary Generation

[0098] The server generates an easy-to-understand summary for users based on the extracted trends and buzzwords. For example, a summary like "Natural language processing and machine learning are important in the latest AI technology trends. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML and stored in a database.

[0099] View Summary

[0100] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[0101] Specific examples

[0102] When a user types "latest AI technology trends" into their device, the server will collect relevant data using the Twitter API and Google News API. This data will undergo preprocessing, morphological analysis, and topic modeling to extract key topics and related trends. Finally, a summary based on the extracted results will be sent to the user's device and displayed. The following is an example of a prompt:

[0103] "Collect relevant data from Twitter and Google News about the latest AI technology trends, extract key trends and buzzwords, and generate summaries."

[0104] As a result, the system of the present invention can quickly and efficiently identify and display the latest information that is of interest to the user.

[0105] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0106] Step 1:

[0107] The user inputs keywords of interest through the terminal, for example, "latest AI technology trends." This input data is sent from the terminal to the server. The input of this process is the keyword entered by the user, and the output is the keyword being sent to the server.

[0108] Step 2:

[0109] The server receives the keywords entered by the user and starts collecting data based on them. Specifically, it calls the Twitter API and Google News API to retrieve the latest tweets and news articles related to "AI technology." The input for this data collection is the keywords entered by the user, and the output is data obtained from social media and news sites.

[0110] Step 3:

[0111] The server pre-processes the collected data, which includes filtering to remove advertisements and spam, normalizing the text, converting uppercase to lowercase, removing special characters, and removing unnecessary duplicate data. The input to this pre-processing is the raw collected data, and the output is the cleaned and formatted data.

[0112] Step 4:

[0113] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Specifically, morphological analysis is performed and the text is tokenized. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. The topics are then grouped using the k-means clustering algorithm. The input to this analysis is the preprocessed data, and the output is a list of topics and related trends.

[0114] Step 5:

[0115] The server extracts key trends and buzzwords based on the results of natural language analysis. Specifically, it identifies frequently occurring keywords and phrases and lists them using the results of topic modeling and clustering. The input for this trend extraction is the analysis results, and the output is a list of frequently occurring keywords and phrases.

[0116] Step 6:

[0117] The server generates a summary for the user based on the extracted trends and buzzwords. A specific example is a summary such as, "The latest AI technology trends are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input for this summary generation are the extracted trends and buzzwords, and the output is the generated summary.

[0118] Step 7:

[0119] The generated summary is sent from the server to the terminal. The terminal receives this summary, parses it, and displays it to the user via a browser or a dedicated application. The user can view the displayed summary and enter new keywords to continue the analysis. The input of this summary display is the generated summary, and the output is the summary displayed on the user's screen.

[0120] (Application example 1)

[0121] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] The explosive growth of information in today's world has made it difficult for users to effectively collect and utilize the latest trending information that matches their interests. Furthermore, information collected using conventional natural language analysis systems and topic modeling techniques often contains noise and cannot fully meet user needs. Furthermore, there is a lack of mechanisms for effectively utilizing user feedback to improve the accuracy of recommendation content. To solve these problems, a system that integrates more advanced data analysis technology and user feedback is needed.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0124] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data based on keywords specified by the user, means for displaying the generated summary on the user's terminal, and means for collecting user feedback to improve the accuracy of recommendation content, thereby enabling users to effectively check the latest information that matches their interests and improving the accuracy of the system's recommendations through feedback.

[0125] "Natural language analysis technology" is a technology for analyzing collected text data and understanding its meaning and structure.

[0126] "Noise" is unnecessary data or information that reduces the accuracy of data analysis.

[0127] "Topic modeling" is a technique for automatically extracting latent topics from collected text data.

[0128] A "clustering algorithm" is an algorithm for dividing data into groups and aggregating highly related data.

[0129] "Latent Dirichlet Allocation" is a topic modeling technique for estimating topic distribution within documents.

[0130] The "k-means algorithm" is a clustering technique for dividing data into k clusters.

[0131] A "summary" is information that extracts the main points from a large amount of information and summarizes them concisely.

[0132] "Feedback" refers to opinions and evaluations provided by users, and is information that is useful for improving the system.

[0133] "Recommendation" is a function that recommends optimal information and content based on the user's interests and preferences.

[0134] A "user terminal" is a device that is directly operated by a user, such as a smartphone or computer.

[0135] This invention is a content recommendation system that provides users with the latest trending information in real time. This system utilizes natural language analysis technology, data preprocessing technology, topic modeling technology, and clustering algorithms. Detailed embodiments of this system are described below.

[0136] Hardware and Software Used

[0137] This system uses the following hardware and software:

[0138] Server: Data collection, preprocessing, analysis, and recommendation processing (Flask, MongoDB)

[0139] Device: Displaying information to users and collecting feedback (smartphone app: React Native, Redux)

[0140] Data collection API: Twitter API, Google News API

[0141] Data preprocessing libraries: pandas, NLTK, BeautifulSoup

[0142] Natural language analysis libraries: spaCy, gensim (LDA), scikit-learn (k-means)

[0143] Data collection

[0144] The server uses the Twitter API or Google News API to collect relevant tweets and news articles based on keywords specified by the user. For example, if the keyword "latest AI technology trends" is entered, the server will collect tweets and news articles related to this keyword.

[0145] Data Preprocessing

[0146] The collected data is preprocessed on the server. During this process, noise is removed from the text data and only the necessary data is extracted. Specific preprocessing steps include removing URLs, removing special characters, normalizing case, and removing unnecessary duplicate data.

[0147] natural language analysis

[0148] Once the data has been preprocessed, the server performs natural language analysis, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[0149] Trend extraction and summary generation

[0150] Based on the analysis, the server identifies frequently occurring keywords and phrases and extracts key trends, which allows it to identify the latest information relevant to the user's interests. It then generates a summary based on the extracted trend information and formats it in HTML.

[0151] View summary and collect feedback

[0152] The generated summary is sent from the server to the user's device. The device parses the summary and displays it to the user via a browser or dedicated application. The user can review the summary and provide feedback. This feedback is sent back to the server and used to improve the accuracy of the recommendations.

[0153] Examples of concrete examples and prompts

[0154] For example, if a user types "latest AI technology trends" into their device, the server will collect data based on the specified keywords and perform preprocessing, natural language analysis, and trend extraction. The resulting summary will be sent to the user's device and displayed in real time. The user can then use this information to enter new keywords and continue with more detailed analysis.

[0155] An example of a prompt for a generative AI model is:

[0156] "Collect news and tweets about the latest AI technology trends and identify key topics."

[0157] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0158] Step 1:

[0159] When a user inputs a keyword into the device, for example, "latest AI technology trends," the device sends this keyword to the server.

[0160] Input: User-specified keywords (e.g., "latest AI technology trends")

[0161] Output: The keyword is sent to the server

[0162] Step 2:

[0163] The server collects data based on the received keywords using the Twitter API and Google News API, calling each API to retrieve related tweets and news articles.

[0164] Input:keyword

[0165] Output: Collected tweets and news articles

[0166] Step 3:

[0167] The server preprocesses the collected data by removing noise from the text data and extracting only the necessary data, including removing URLs, special characters, case normalization, and removing unnecessary duplicate data.

[0168] Input: Collected tweets and news articles

[0169] Output: Preprocessed text data

[0170] Step 4:

[0171] The server performs natural language analysis on the preprocessed data, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[0172] Input: Preprocessed text data

[0173] Output: Topics and clustering results

[0174] Step 5:

[0175] The server uses the results of the analysis to identify frequently occurring keywords and phrases, extract key trends, and generate summaries based on this information, formatting them in HTML.

[0176] Input: Topics and clustering results

[0177] Output: Generated summary

[0178] Step 6:

[0179] The generated summary is sent from the server to the user's device, which parses it and displays it to the user via a browser or dedicated application.

[0180] Input: Generated summary

[0181] Output: A summary displayed on the terminal

[0182] Step 7:

[0183] The user reviews the displayed summary and provides feedback, which the device sends to the server.

[0184] Input: User feedback

[0185] Output: Feedback sent to the server

[0186] Step 8:

[0187] The server analyzes the received feedback to improve the accuracy of the recommendations, and appropriate trend information based on the feedback is reflected in future recommendations.

[0188] Input: User feedback

[0189] Output: Improved recommendation content

[0190] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0191] This invention is a data analysis system that can grasp trends and buzzwords in real time. It combines an emotion engine with a system that automatically collects and analyzes data and extracts information from news and social media. Each element of this system is explained below.

[0192] Data collection

[0193] Based on keywords entered by the user into the device, the server collects data using the APIs of multiple social media platforms and news sites. For example, the server may use a social media API to obtain the latest posts about "AI technology" and a news site API to collect related articles. The collected data is then stored in the server's database.

[0194] Data Preprocessing

[0195] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, and removes duplicate data.

[0196] Natural language analysis (NLP) technology

[0197] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., LDA) is then used to identify major topics. Related topics are then grouped together using a clustering algorithm (e.g., k-means clustering).

[0198] Trend and buzzword extraction

[0199] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[0200] Emotion engine integration

[0201] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion such as "surprised," the emotion engine will recognize that emotion and provide information based on that emotion.

[0202] Summary Generation

[0203] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[0204] Sending and viewing summaries

[0205] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time, enter new topics or keywords as needed, and request another analysis.

[0206] Specific examples

[0207] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary provided. Finally, the generated summary is sent to the user's device and can be viewed in real time. This process allows users to obtain more comprehensive information quickly.

[0208] The processing flow will be explained below.

[0209] Step 1:

[0210] User enters topic or keyword

[0211] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[0212] Step 2:

[0213] The server analyzes the user's input and recognizes their emotions.

[0214] The server analyzes the received input using an emotion engine to recognize the user's emotions (e.g., "surprise," "interest," "concern," etc.). The analysis results are used for subsequent data analysis and summary generation.

[0215] Step 3:

[0216] Server collects data from social media and news sites

[0217] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and the collected data is stored in the server's local database.

[0218] Step 4:

[0219] The server preprocesses the data

[0220] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[0221] Step 5:

[0222] The server performs natural language processing (NLP)

[0223] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[0224] Step 6:

[0225] Trend and buzzword extraction

[0226] The server identifies frequently occurring keywords and phrases from the NLP analysis results. Based on the results of topic modeling and clustering, it lists major trends and buzzwords. The results of the sentiment engine are also taken into account at this stage.

[0227] Step 7:

[0228] Summary Generation

[0229] The server generates a summary for the user based on the extracted trends and buzzwords and the results of the emotion engine analysis. For example, it might say, "The latest trends in AI technology that you are interested in are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are attracting attention." The generated summary is formatted in HTML.

[0230] Step 8:

[0231] Sending a summary

[0232] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[0233] Step 9:

[0234] The device displays a summary to the user

[0235] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[0236] Specific examples

[0237] 1. User Input

[0238] The user types "latest AI technology trends" into the input field on their device.

[0239] The terminal sends the entered keyword to the server.

[0240] 2. Emotion analysis

[0241] The server analyzes the user's input using an emotion engine and recognizes their "interests and concerns."

[0242] 3. Data Collection

[0243] The server uses the Twitter API and Google News API to collect the latest data related to "AI technology."

[0244] 4. Data Preprocessing

[0245] The server uses filtering algorithms to remove advertisements and spam and normalize the text.

[0246] 5. Natural language analysis

[0247] The server performs morphological analysis and performs topic modeling and clustering.

[0248] 6. Trend Extraction

[0249] The server extracts frequently appearing keywords (e.g., "GPT-4," "BERT," and "deep learning") from the analysis results.

[0250] 7. Summary Generation

[0251] The server takes into account the analysis results of the emotion engine and generates a summary that reads, "The latest trends in AI technology are natural language processing and machine learning. GPT-4 and BERT in particular are attracting attention."

[0252] 8. Summary Transmission

[0253] The server sends the generated summary to the terminal.

[0254] 9. Summary View

[0255] The user views the summary received on the device and requests further analysis using new topics or keywords.

[0256] This process allows users to receive the latest trend information quickly, emotionally, and appropriately, helping them make decisions.

[0257] Example 2

[0258] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0259] Conventional data analysis systems have difficulty grasping trends and buzzwords that correspond to user interests in real time. Furthermore, because sentiment analysis is not integrated, they are unable to provide information that corresponds to the user's emotions and are unable to generate appropriate summaries. As a result, users are unable to obtain information quickly and accurately, resulting in lower search accuracy and lower satisfaction.

[0260] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0261] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data from multiple sources based on user-specified keywords, means for integrating a sentiment analysis engine to understand user sentiment and adjust the generated summary based on the sentiment, and means for transmitting the generated summary to the user's terminal and displaying it. This makes it possible to provide the latest information according to the user's interests in real time and generate appropriate summaries that take the user's sentiment into consideration.

[0262] "Natural language analysis technology" is a technology that analyzes text data and grasps meaning and structure through morphological analysis, topic modeling, clustering, etc.

[0263] "Noise" refers to unnecessary information or data in data analysis, including advertisements, spam, and duplicate data.

[0264] "Topic modeling" is a method for extracting latent topics from large amounts of text data, and a representative algorithm is Latent Dirichlet Allocation (LDA).

[0265] A "clustering algorithm" is a method for grouping data based on similar characteristics, and a representative algorithm is the k-means algorithm.

[0266] "Trends" refer to subjects or keywords that are frequently mentioned during a particular period of time.

[0267] A "buzzword" is a keyword or phrase that rapidly gains attention in a short period of time.

[0268] "User-specified keywords" are words or phrases that a user inputs into the system to be used as search and analysis criteria.

[0269] An "emotion analysis engine" is a system that analyzes emotions contained in text data and recognizes emotional states such as positive, negative, and surprise.

[0270] A "summary" is summary information generated based on the analysis results, and is information in a format that is organized so that the user can easily understand it.

[0271] A "terminal" is a hardware device that allows a user to communicate with the system through an interface, and includes a personal computer, a smartphone, etc.

[0272] The present invention is a data analysis system that can grasp trends and buzzwords in real time, and combines an emotion engine with a system in which a server automatically collects and analyzes data and extracts information from news and social media. Details of an embodiment of this system are provided below.

[0273] Data collection

[0274] The user enters a keyword into the device. For example, the user enters "latest AI technology trends." Based on this keyword, the server collects data using multiple social media APIs (e.g., Twitter API) and news site APIs (e.g., News API). In this process, the server collects posts and articles related to the specified keyword. The collected data is stored in the server's database (e.g., MongoDB or MySQL (registered trademark)).

[0275] Data Preprocessing

[0276] The collected data is pre-processed to remove noise: the server uses filtering algorithms (e.g., regular expression filters) to remove advertisements and spam, and normalizes the text, converting it to lowercase, removing special characters, and deleting duplicate data.

[0277] Natural language analysis (NLP)

[0278] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation, LDA) is then used to identify major topics, followed by a clustering algorithm (e.g., k-means clustering) to group related topics.

[0279] Trend and buzzword extraction

[0280] Based on the results of natural language analysis, the server identifies frequently occurring keywords and phrases. Based on the results of topic modeling and clustering, it lists major trends and buzzwords, thereby identifying the latest information that matches the user's interests.

[0281] Emotion engine integration

[0282] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion, such as "surprised," the emotion engine (e.g., VADemotion) will recognize that emotion and adjust the information provided accordingly.

[0283] Summary Generation

[0284] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[0285] Sending and viewing summaries

[0286] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time. If necessary, they can also input new topics or keywords and request another analysis.

[0287] Specific examples

[0288] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary content provided. Finally, the generated summary is sent to the user's device and can be viewed in real time.

[0289] Prompt Sentence Examples

[0290] "What are the latest trends in AI technology?"

[0291] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0292] Step 1:

[0293] The user inputs a keyword into the device. An example input is "latest AI technology trends." This input keyword becomes the basis for the subsequent data collection process. The input keyword is sent to the server.

[0294] Step 2:

[0295] Based on the keywords received from the user, the server calls social media APIs (e.g., Twitter API) and news site APIs (e.g., News API) to collect related data. Specifically, the server sends a query using the keywords to each API, and the retrieved data is returned to the server in JSON format. This data includes related posts and articles.

[0296] Step 3:

[0297] The server stores the collected data in a database (e.g., MongoDB or MySQL). A database is used to efficiently organize the data and store it in a searchable format. When storing, the JSON data is split into fields based on the appropriate schema and inserted.

[0298] Step 4:

[0299] The server performs pre-processing on the collected data. In this step, regular expression filters are used to remove advertisements and spam, and text normalization is performed. Specific operations include converting uppercase to lowercase, removing special characters, and deleting duplicates. The input is raw data retrieved from the database, and the output is the pre-processed, clean data.

[0300] Step 5:

[0301] The server analyzes the preprocessed data using natural language analysis techniques (e.g., morphological analysis, topic modeling, clustering). First, morphological analysis is performed to tokenize the text. Next, Latent Dirichlet Allocation (LDA) is used to identify major topics. Next, k-means clustering is used to group related topics. The input is the preprocessed data, and the output is a list of topics and clusters resulting from the analysis.

[0302] Step 6:

[0303] The server identifies frequently occurring keywords and phrases based on the results of natural language analysis and extracts major trends and buzzwords. In this step, important keywords are extracted and listed based on the results of topic modeling and clustering. The input is the analysis results, and the output is a list of extracted major trends and buzzwords.

[0304] Step 7:

[0305] The server uses an emotion analysis engine (e.g., VADemotion) to analyze the user's input and recognize their emotions. Specifically, when a user enters a phrase that expresses an emotion, such as "surprised," the server identifies that emotion and generates emotion data as the analysis result. The input is text related to the user's emotional mood, and the output is the recognized emotion data.

[0306] Step 8:

[0307] The server takes into account the results of the sentiment analysis and generates a summary for the user. Specifically, it formats the summary into something like, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML. The input is the extracted trends, buzzwords, and sentiment data, and the output is the formatted summary.

[0308] Step 9:

[0309] The server sends the generated summary to the terminal. Specifically, the server sends summary data to the user's terminal using HTTP or the WebSocket protocol. The input is the summary in HTML format, and the output is the summary data received by the terminal.

[0310] Step 10:

[0311] The device parses the received summary and displays it to the user via a browser or dedicated application. Specifically, the device renders the received data in HTML format and displays it to the user. Through this interface, the user can check trends and buzzwords in real time. The user can also input new topics or keywords and request analysis again. The input is the received summary data, and the output is the information displayed to the user.

[0312] (Application example 2)

[0313] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] In today's information society, it is difficult for users to efficiently gather and understand the latest trend information. In particular, there is a need for a method to extract appropriate information from a wide range of information sources, such as social media and news sites, and provide it according to the user's interests. However, doing this manually takes a lot of time and effort. In addition, a system is needed that can provide more useful information by taking the user's emotions into consideration rather than simply collecting information.

[0315] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for performing sentiment analysis based on the analyzed trends and buzzwords, means for providing the user with a summary generated in consideration of the sentiment analysis, and means for displaying the generated summary on the user's terminal. This allows the user to efficiently obtain the latest trend information, and because the information is based on the user's emotions, more useful and comprehensive information can be provided.

[0316] "Natural language analysis technology" is a technology that uses computers to analyze text data and understand language structure.

[0317] "Noise" refers to unnecessary data or information that can cause inaccurate analysis results.

[0318] "Topic modeling" is a technique for extracting themes and themes from text data, and is a technology that analyzes data to identify potential topics.

[0319] A "clustering algorithm" is a mathematical technique for classifying data into groups, grouping related data points together.

[0320] "Sentiment analysis" is a technology that automatically extracts and analyzes user emotions and opinions from text data.

[0321] A "summary" summarizes the analyzed information and summarizes the main points.

[0322] "User terminal" refers to a communication device, such as a smartphone, tablet, or PC, that a user uses to obtain information.

[0323] This invention provides a system that efficiently collects real-time trend information based on keywords specified by a user, generates summaries taking into account sentiment analysis, and displays the summaries on the user's terminal. Specific embodiments of the system are described below.

[0324] System Configuration

[0325] This system is configured using the following hardware and software:

[0326] Hardware: Servers, user devices (smartphones, tablets, PCs)

[0327] Software: APIs (news site APIs, social media APIs), databases (MySQL, MongoDB), programming languages ​​(Python, JavaScript (registered trademark)), NLP libraries (spaCy, NLTK), cloud services (AWS (registered trademark), Google Cloud)

[0328] Data collection

[0329] The server uses APIs of news sites and social media to collect related data based on keywords entered by the user on the device. This data includes text, images, videos, etc. For example, if a user enters "latest AI technology trends," the server retrieves the latest posts and articles related to "AI technology."

[0330] Data Preprocessing

[0331] The collected data is pre-processed to remove noise. Specifically, filtering algorithms are used to remove advertisements and spam, and text normalization is performed (for example, converting uppercase letters to lowercase and removing special characters). Duplicate data is also removed to improve the accuracy of the analysis.

[0332] Natural Language Analysis and Trend Extraction

[0333] After preprocessing, the data is tokenized using natural language analysis, topic modeling (e.g., Latent Dirichlet Allocation) is used to identify major topics, and then a clustering algorithm (e.g., k-means clustering) is used to group related topics and extract trends and buzzwords.

[0334] Emotion analysis

[0335] Based on the extracted trends and buzzwords, an emotion engine is used to analyze the user's emotions, thereby determining the emotions associated with the keywords entered by the user.

[0336] Summary generation and display

[0337] Taking into account the results of sentiment analysis, a summary optimized for the user is generated. For example, it may be formatted as, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." The generated summary is formatted in HTML and sent to the user's device. Users can check trend information that changes in real time and request reanalysis by entering new topics or keywords.

[0338] Specific examples

[0339] For example, if a user types "latest AI technology trends" into their device, the server will instantly collect relevant data from social media and news sites, perform sentiment analysis through the steps described above, and generate an optimal summary. Through this series of processes, users can efficiently obtain the latest trend information.

[0340] Prompt Sentence Examples

[0341] Collect data on the latest AI technology trends, perform sentiment analysis, and generate summaries based on user sentiment.

[0342] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0343] Step 1:

[0344] The user enters a trend keyword into the device. This entered keyword is sent to the server. For example, the user enters "latest AI technology trends." This keyword will become the basis for data collection in the next step.

[0345] Step 2:

[0346] The server collects related data using news site APIs and social media APIs based on the received keywords. Specifically, it retrieves posts and articles containing the user-entered "latest AI technology trends" via the API and stores the data in a database. The input is the keywords, and the output is the collected text data and metadata.

[0347] Step 3:

[0348] The server performs preprocessing to remove noise from the data stored in the database. Specifically, it uses filtering algorithms to remove advertisements and spam, and normalizes the text. At this stage, uppercase and lowercase letters are converted, special characters are removed, and duplicate data is deleted. The input is the collected text data, and the output is the preprocessed, clean text data.

[0349] Step 4:

[0350] The server tokenizes the preprocessed data using natural language processing (NLP) techniques, identifies major topics using topic modeling (e.g., Latent Dirichlet Allocation), and then groups related topics using a clustering algorithm (e.g., k-means clustering). The input is the preprocessed, clean text data, and the output is data for each topic and cluster.

[0351] Step 5:

[0352] Based on the topic and cluster information, the server analyzes the data using a sentiment analysis engine, which identifies user sentiment toward each topic or buzzword. Specifically, the server analyzes the text data and classifies it into sentiment categories such as positive, negative, and neutral. The input is data for each topic and cluster, and the output is the sentiment analysis results.

[0353] Step 6:

[0354] Based on the results of the sentiment analysis, the server generates a summary optimized for the user. For example, if a user is interested in "the latest AI technology trends," the server generates a summary such as, "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input is the sentiment analysis results and topic information, and the output is a formatted summary.

[0355] Step 7:

[0356] The server sends the generated summary to the user's device, which then displays the summary. The user can check trend information changing in real time and can also input new keywords to request another analysis. The input is the generated summary, and the output is the summary displayed on the user's device.

[0357] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0358] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0359] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0360] [Second embodiment]

[0361] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0362] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0363] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0364] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0365] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0366] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0367] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0368] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0369] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0370] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0371] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0372] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0373] This invention is a data analysis system that can grasp trends and buzzwords in real time, and performs automatic data collection and analysis to extract information from news and social media. Each element that makes up this system is explained below.

[0374] Data collection

[0375] The server collects data using the APIs of multiple social media platforms and news sites based on keywords entered by the user on their device. For example, it uses the Twitter API to obtain the latest tweets about "AI technology." It also uses the Google News API to collect related news articles. The collected data is stored in the server's database.

[0376] Data Preprocessing

[0377] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, etc., and removes unnecessary duplicate data.

[0378] Natural language analysis (NLP) technology

[0379] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation - LDA) is then used to identify major topics. After this, a clustering algorithm (e.g., k-means clustering) is used to group related topics.

[0380] Trend and buzzword extraction

[0381] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[0382] Summary Generation

[0383] Based on the extracted trends and buzzwords, the server generates an easy-to-understand summary for the user. For example, a summary like "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." This summary is formatted in HTML.

[0384] View Summary

[0385] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[0386] Specific examples

[0387] When a user types "latest AI technology trends" into their device, the server collects related data using the Twitter API and Google News API. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. A summary generated based on the extraction results is sent to the user's device and displayed. The user can view this information in real time and continue the analysis by entering new topics and keywords.

[0388] In this way, the system of the present invention utilizes natural language analysis technology to extract, analyze, and display important trends and buzzwords from vast amounts of data in real time, thereby supporting users' knowledge gathering and decision-making.

[0389] The processing flow will be explained below.

[0390] Step 1:

[0391] User enters topic or keyword

[0392] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[0393] Step 2:

[0394] Server collects data from social media and news sites

[0395] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and stores the collected data in a local database.

[0396] Step 3:

[0397] The server preprocesses the data

[0398] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[0399] Step 4:

[0400] The server performs natural language processing (NLP)

[0401] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[0402] Step 5:

[0403] Trend and buzzword extraction

[0404] The server identifies frequently occurring keywords and phrases from the NLP analysis results, and lists major trends and buzzwords based on the results of topic modeling and clustering.

[0405] Step 6:

[0406] The server generates the summary

[0407] The server generates a summary for the user based on the extracted trends and buzzwords. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular."

[0408] Step 7:

[0409] The server sends the summary to the device

[0410] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[0411] Step 8:

[0412] The device displays a summary to the user

[0413] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[0414] Example 1

[0415] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0416] It solves the difficulty of quickly and effectively identifying trends and buzzwords from vast amounts of data and providing information that interests users in real time.

[0417] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0418] In this invention, the server includes means for collecting data based on keywords specified by a user, means for removing noise from the collected data, means for preprocessing the collected data, means for analyzing the preprocessed data using natural language analysis techniques, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for generating summaries based on the extracted trends and buzzwords, and means for displaying the generated summaries on the user's terminal, thereby enabling the latest information of interest to the user to be quickly and efficiently identified and displayed.

[0419] "Means for collecting data based on user-specified keywords" is a function that allows a user to input a specific keyword of interest and obtain information related to that keyword from multiple sources.

[0420] "Means for removing noise from collected data" refers to a processing function for removing unnecessary information, advertisements, and spam messages from collected data and improving the quality of the data.

[0421] "Means for preprocessing collected data" refers to a function for formatting collected data into a format that is easy to analyze, such as converting uppercase letters to lowercase, removing special characters, and deleting unnecessary duplicate data.

[0422] "Means for analyzing preprocessed data using natural language processing technology" refers to a function that utilizes natural language processing technology to further analyze preprocessed data and divide it into meaningful units. Specifically, this includes morphological analysis and topic modeling.

[0423] "Means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms" refers to the function of identifying important topics and frequently occurring keywords and phrases from collected and preprocessed data and grouping them. Latent Dirichlet Allocation is used for topic modeling, and the k-means algorithm is used for clustering.

[0424] "Means for generating summaries based on extracted trends and buzzwords" is a function for generating easy-to-understand summaries for users based on important information identified from the analyzed data.

[0425] The "means for displaying the generated summary on the user's terminal" is a function for sending the summary information generated by the server to the user's terminal and displaying it to the user via a browser or a dedicated application.

[0426] MODE FOR CARRYING OUT THE INVENTION

[0427] This invention relates to a data analysis system that extracts and displays in real time the latest trends and buzzwords that users are interested in. This system has the ability to automatically collect and analyze data and obtain information related to specific keywords from multiple sources, such as social media and news sites.

[0428] Data collection

[0429] The server collects data using APIs of social media and news sites based on keywords entered by the user on the device. For example, the Twitter API or Google News API can be used. A specific example is collecting the latest tweets and news articles related to "AI technology." The collected data is stored in the server's database.

[0430] Data Preprocessing

[0431] The collected data is pre-processed by the server. Filtering algorithms are used to remove advertisements and spam, and text data is normalized. This includes converting uppercase to lowercase, removing special characters, and deleting unnecessary duplicate data. The pre-processed data is then stored in a clean database.

[0432] Natural language analysis (NLP) technology

[0433] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Morphological analysis is performed to tokenize the text. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. Furthermore, the k-means clustering algorithm is used to group related topics to extract important trends and buzzwords.

[0434] Summary Generation

[0435] The server generates an easy-to-understand summary for users based on the extracted trends and buzzwords. For example, a summary like "Natural language processing and machine learning are important in the latest AI technology trends. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML and stored in a database.

[0436] View Summary

[0437] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[0438] Specific examples

[0439] When a user types "latest AI technology trends" into their device, the server will collect relevant data using the Twitter API and Google News API. This data will undergo preprocessing, morphological analysis, and topic modeling to extract key topics and related trends. Finally, a summary based on the extracted results will be sent to the user's device and displayed. The following is an example of a prompt:

[0440] "Collect relevant data from Twitter and Google News about the latest AI technology trends, extract key trends and buzzwords, and generate summaries."

[0441] As a result, the system of the present invention can quickly and efficiently identify and display the latest information that is of interest to the user.

[0442] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0443] Step 1:

[0444] The user inputs keywords of interest through the terminal, for example, "latest AI technology trends." This input data is sent from the terminal to the server. The input of this process is the keyword entered by the user, and the output is the keyword being sent to the server.

[0445] Step 2:

[0446] The server receives the keywords entered by the user and starts collecting data based on them. Specifically, it calls the Twitter API and Google News API to retrieve the latest tweets and news articles related to "AI technology." The input for this data collection is the keywords entered by the user, and the output is data obtained from social media and news sites.

[0447] Step 3:

[0448] The server pre-processes the collected data, which includes filtering to remove advertisements and spam, normalizing the text, converting uppercase to lowercase, removing special characters, and removing unnecessary duplicate data. The input to this pre-processing is the raw collected data, and the output is the cleaned and formatted data.

[0449] Step 4:

[0450] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Specifically, morphological analysis is performed and the text is tokenized. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. The topics are then grouped using the k-means clustering algorithm. The input to this analysis is the preprocessed data, and the output is a list of topics and related trends.

[0451] Step 5:

[0452] The server extracts key trends and buzzwords based on the results of natural language analysis. Specifically, it identifies frequently occurring keywords and phrases and lists them using the results of topic modeling and clustering. The input for this trend extraction is the analysis results, and the output is a list of frequently occurring keywords and phrases.

[0453] Step 6:

[0454] The server generates a summary for the user based on the extracted trends and buzzwords. A specific example is a summary such as, "The latest AI technology trends are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input for this summary generation are the extracted trends and buzzwords, and the output is the generated summary.

[0455] Step 7:

[0456] The generated summary is sent from the server to the terminal. The terminal receives this summary, parses it, and displays it to the user via a browser or a dedicated application. The user can view the displayed summary and enter new keywords to continue the analysis. The input of this summary display is the generated summary, and the output is the summary displayed on the user's screen.

[0457] (Application example 1)

[0458] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0459] The explosive growth of information in today's world has made it difficult for users to effectively collect and utilize the latest trending information that matches their interests. Furthermore, information collected using conventional natural language analysis systems and topic modeling techniques often contains noise and cannot fully meet user needs. Furthermore, there is a lack of mechanisms for effectively utilizing user feedback to improve the accuracy of recommendation content. To solve these problems, a system that integrates more advanced data analysis technology and user feedback is needed.

[0460] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0461] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data based on keywords specified by the user, means for displaying the generated summary on the user's terminal, and means for collecting user feedback to improve the accuracy of recommendation content, thereby enabling users to effectively check the latest information that matches their interests and improving the accuracy of the system's recommendations through feedback.

[0462] "Natural language analysis technology" is a technology for analyzing collected text data and understanding its meaning and structure.

[0463] "Noise" is unnecessary data or information that reduces the accuracy of data analysis.

[0464] "Topic modeling" is a technique for automatically extracting latent topics from collected text data.

[0465] A "clustering algorithm" is an algorithm for dividing data into groups and aggregating highly related data.

[0466] "Latent Dirichlet Allocation" is a topic modeling technique for estimating topic distribution within documents.

[0467] The "k-means algorithm" is a clustering technique for dividing data into k clusters.

[0468] A "summary" is information that extracts the main points from a large amount of information and summarizes them concisely.

[0469] "Feedback" refers to opinions and evaluations provided by users, and is information that is useful for improving the system.

[0470] "Recommendation" is a function that recommends optimal information and content based on the user's interests and preferences.

[0471] A "user terminal" is a device that is directly operated by a user, such as a smartphone or computer.

[0472] This invention is a content recommendation system that provides users with the latest trending information in real time. This system utilizes natural language analysis technology, data preprocessing technology, topic modeling technology, and clustering algorithms. Detailed embodiments of this system are described below.

[0473] Hardware and Software Used

[0474] This system uses the following hardware and software:

[0475] Server: Data collection, preprocessing, analysis, and recommendation processing (Flask, MongoDB)

[0476] Device: Displaying information to users and collecting feedback (smartphone app: React Native, Redux)

[0477] Data collection API: Twitter API, Google News API

[0478] Data preprocessing libraries: pandas, NLTK, BeautifulSoup

[0479] Natural language analysis libraries: spaCy, gensim (LDA), scikit-learn (k-means)

[0480] Data collection

[0481] The server uses the Twitter API or Google News API to collect relevant tweets and news articles based on keywords specified by the user. For example, if the keyword "latest AI technology trends" is entered, the server will collect tweets and news articles related to this keyword.

[0482] Data Preprocessing

[0483] The collected data is preprocessed on the server. During this process, noise is removed from the text data and only the necessary data is extracted. Specific preprocessing steps include removing URLs, removing special characters, normalizing case, and removing unnecessary duplicate data.

[0484] natural language analysis

[0485] Once the data has been preprocessed, the server performs natural language analysis, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[0486] Trend extraction and summary generation

[0487] Based on the analysis, the server identifies frequently occurring keywords and phrases and extracts key trends, which allows it to identify the latest information relevant to the user's interests. It then generates a summary based on the extracted trend information and formats it in HTML.

[0488] View summary and collect feedback

[0489] The generated summary is sent from the server to the user's device. The device parses the summary and displays it to the user via a browser or dedicated application. The user can review the summary and provide feedback. This feedback is sent back to the server and used to improve the accuracy of the recommendations.

[0490] Examples of concrete examples and prompts

[0491] For example, if a user types "latest AI technology trends" into their device, the server will collect data based on the specified keywords and perform preprocessing, natural language analysis, and trend extraction. The resulting summary will be sent to the user's device and displayed in real time. The user can then use this information to enter new keywords and continue with more detailed analysis.

[0492] An example of a prompt for a generative AI model is:

[0493] "Collect news and tweets about the latest AI technology trends and identify key topics."

[0494] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0495] Step 1:

[0496] When a user inputs a keyword into the device, for example, "latest AI technology trends," the device sends this keyword to the server.

[0497] Input: User-specified keywords (e.g., "latest AI technology trends")

[0498] Output: The keyword is sent to the server

[0499] Step 2:

[0500] The server collects data based on the received keywords using the Twitter API and Google News API, calling each API to retrieve related tweets and news articles.

[0501] Input:keyword

[0502] Output: Collected tweets and news articles

[0503] Step 3:

[0504] The server preprocesses the collected data by removing noise from the text data and extracting only the necessary data, including removing URLs, special characters, case normalization, and removing unnecessary duplicate data.

[0505] Input: Collected tweets and news articles

[0506] Output: Preprocessed text data

[0507] Step 4:

[0508] The server performs natural language analysis on the preprocessed data, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[0509] Input: Preprocessed text data

[0510] Output: Topics and clustering results

[0511] Step 5:

[0512] The server uses the results of the analysis to identify frequently occurring keywords and phrases, extract key trends, and generate summaries based on this information, formatting them in HTML.

[0513] Input: Topics and clustering results

[0514] Output: Generated summary

[0515] Step 6:

[0516] The generated summary is sent from the server to the user's device, which parses it and displays it to the user via a browser or dedicated application.

[0517] Input: Generated summary

[0518] Output: A summary displayed on the terminal

[0519] Step 7:

[0520] The user reviews the displayed summary and provides feedback, which the device sends to the server.

[0521] Input: User feedback

[0522] Output: Feedback sent to the server

[0523] Step 8:

[0524] The server analyzes the received feedback to improve the accuracy of the recommendations, and appropriate trend information based on the feedback is reflected in future recommendations.

[0525] Input: User feedback

[0526] Output: Improved recommendation content

[0527] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0528] This invention is a data analysis system that can grasp trends and buzzwords in real time. It combines an emotion engine with a system that automatically collects and analyzes data and extracts information from news and social media. Each element of this system is explained below.

[0529] Data collection

[0530] Based on keywords entered by the user into the device, the server collects data using the APIs of multiple social media platforms and news sites. For example, the server may use a social media API to obtain the latest posts about "AI technology" and a news site API to collect related articles. The collected data is then stored in the server's database.

[0531] Data Preprocessing

[0532] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, and removes duplicate data.

[0533] Natural language analysis (NLP) technology

[0534] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., LDA) is then used to identify major topics. Related topics are then grouped together using a clustering algorithm (e.g., k-means clustering).

[0535] Trend and buzzword extraction

[0536] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[0537] Emotion engine integration

[0538] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion such as "surprised," the emotion engine will recognize that emotion and provide information based on that emotion.

[0539] Summary Generation

[0540] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[0541] Sending and viewing summaries

[0542] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time, enter new topics or keywords as needed, and request another analysis.

[0543] Specific examples

[0544] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary provided. Finally, the generated summary is sent to the user's device and can be viewed in real time. This process allows users to obtain more comprehensive information quickly.

[0545] The processing flow will be explained below.

[0546] Step 1:

[0547] User enters topic or keyword

[0548] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[0549] Step 2:

[0550] The server analyzes the user's input and recognizes their emotions.

[0551] The server analyzes the received input using an emotion engine to recognize the user's emotions (e.g., "surprise," "interest," "concern," etc.). The analysis results are used for subsequent data analysis and summary generation.

[0552] Step 3:

[0553] Server collects data from social media and news sites

[0554] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and the collected data is stored in the server's local database.

[0555] Step 4:

[0556] The server preprocesses the data

[0557] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[0558] Step 5:

[0559] The server performs natural language processing (NLP)

[0560] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[0561] Step 6:

[0562] Trend and buzzword extraction

[0563] The server identifies frequently occurring keywords and phrases from the NLP analysis results. Based on the results of topic modeling and clustering, it lists major trends and buzzwords. The results of the sentiment engine are also taken into account at this stage.

[0564] Step 7:

[0565] Summary Generation

[0566] The server generates a summary for the user based on the extracted trends and buzzwords and the results of the emotion engine analysis. For example, it might say, "The latest trends in AI technology that you are interested in are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are attracting attention." The generated summary is formatted in HTML.

[0567] Step 8:

[0568] Sending a summary

[0569] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[0570] Step 9:

[0571] The device displays a summary to the user

[0572] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[0573] Specific examples

[0574] 1. User Input

[0575] The user types "latest AI technology trends" into the input field on their device.

[0576] The terminal sends the entered keyword to the server.

[0577] 2. Emotion analysis

[0578] The server analyzes the user's input using an emotion engine and recognizes their "interests and concerns."

[0579] 3. Data Collection

[0580] The server uses the Twitter API and Google News API to collect the latest data related to "AI technology."

[0581] 4. Data Preprocessing

[0582] The server uses filtering algorithms to remove advertisements and spam and normalize the text.

[0583] 5. Natural language analysis

[0584] The server performs morphological analysis and performs topic modeling and clustering.

[0585] 6. Trend Extraction

[0586] The server extracts frequently appearing keywords (e.g., "GPT-4," "BERT," and "deep learning") from the analysis results.

[0587] 7. Summary Generation

[0588] The server takes into account the analysis results of the emotion engine and generates a summary that reads, "The latest trends in AI technology are natural language processing and machine learning. GPT-4 and BERT in particular are attracting attention."

[0589] 8. Summary Transmission

[0590] The server sends the generated summary to the terminal.

[0591] 9. Summary View

[0592] The user views the summary received on the device and requests further analysis using new topics or keywords.

[0593] This process allows users to receive the latest trend information quickly, emotionally, and appropriately, helping them make decisions.

[0594] Example 2

[0595] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0596] Conventional data analysis systems have difficulty grasping trends and buzzwords that correspond to user interests in real time. Furthermore, because sentiment analysis is not integrated, they are unable to provide information that corresponds to the user's emotions and are unable to generate appropriate summaries. As a result, users are unable to obtain information quickly and accurately, resulting in lower search accuracy and lower satisfaction.

[0597] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0598] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data from multiple sources based on user-specified keywords, means for integrating a sentiment analysis engine to understand user sentiment and adjust the generated summary based on the sentiment, and means for transmitting the generated summary to the user's terminal and displaying it. This makes it possible to provide the latest information according to the user's interests in real time and generate appropriate summaries that take the user's sentiment into consideration.

[0599] "Natural language analysis technology" is a technology that analyzes text data and grasps meaning and structure through morphological analysis, topic modeling, clustering, etc.

[0600] "Noise" refers to unnecessary information or data in data analysis, including advertisements, spam, and duplicate data.

[0601] "Topic modeling" is a method for extracting latent topics from large amounts of text data, and a representative algorithm is Latent Dirichlet Allocation (LDA).

[0602] A "clustering algorithm" is a method for grouping data based on similar characteristics, and a representative algorithm is the k-means algorithm.

[0603] "Trends" refer to subjects or keywords that are frequently mentioned during a particular period of time.

[0604] A "buzzword" is a keyword or phrase that rapidly gains attention in a short period of time.

[0605] "User-specified keywords" are words or phrases that a user inputs into the system to be used as search and analysis criteria.

[0606] An "emotion analysis engine" is a system that analyzes emotions contained in text data and recognizes emotional states such as positive, negative, and surprise.

[0607] A "summary" is summary information generated based on the analysis results, and is information in a format that is organized so that the user can easily understand it.

[0608] A "terminal" is a hardware device that allows a user to communicate with the system through an interface, and includes a personal computer, a smartphone, etc.

[0609] The present invention is a data analysis system that can grasp trends and buzzwords in real time, and combines an emotion engine with a system in which a server automatically collects and analyzes data and extracts information from news and social media. Details of an embodiment of this system are provided below.

[0610] Data collection

[0611] The user enters a keyword into the device. For example, they might enter "latest AI technology trends." Based on this keyword, the server collects data using multiple social media APIs (e.g., Twitter API) and news site APIs (e.g., News API). In doing so, the server collects posts and articles related to the specified keyword. The collected data is stored in the server's database (e.g., MongoDB or MySQL).

[0612] Data Preprocessing

[0613] The collected data is pre-processed to remove noise: the server uses filtering algorithms (e.g., regular expression filters) to remove advertisements and spam, and normalizes the text, converting it to lowercase, removing special characters, and deleting duplicate data.

[0614] Natural language analysis (NLP)

[0615] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation, LDA) is then used to identify major topics, followed by a clustering algorithm (e.g., k-means clustering) to group related topics.

[0616] Trend and buzzword extraction

[0617] Based on the results of natural language analysis, the server identifies frequently occurring keywords and phrases. Based on the results of topic modeling and clustering, it lists major trends and buzzwords, thereby identifying the latest information that matches the user's interests.

[0618] Emotion engine integration

[0619] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion, such as "surprised," the emotion engine (e.g., VADemotion) will recognize that emotion and adjust the information provided accordingly.

[0620] Summary Generation

[0621] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[0622] Sending and viewing summaries

[0623] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time. If necessary, they can also input new topics or keywords and request another analysis.

[0624] Specific examples

[0625] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary content provided. Finally, the generated summary is sent to the user's device and can be viewed in real time.

[0626] Prompt Sentence Examples

[0627] "What are the latest trends in AI technology?"

[0628] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0629] Step 1:

[0630] The user inputs a keyword into the device. An example input is "latest AI technology trends." This input keyword becomes the basis for the subsequent data collection process. The input keyword is sent to the server.

[0631] Step 2:

[0632] Based on the keywords received from the user, the server calls social media APIs (e.g., Twitter API) and news site APIs (e.g., News API) to collect related data. Specifically, the server sends a query using the keywords to each API, and the retrieved data is returned to the server in JSON format. This data includes related posts and articles.

[0633] Step 3:

[0634] The server stores the collected data in a database (e.g., MongoDB or MySQL). A database is used to efficiently organize the data and store it in a searchable format. When storing, the JSON data is split into fields based on the appropriate schema and inserted.

[0635] Step 4:

[0636] The server performs pre-processing on the collected data. In this step, regular expression filters are used to remove advertisements and spam, and text normalization is performed. Specific operations include converting uppercase to lowercase, removing special characters, and deleting duplicates. The input is raw data retrieved from the database, and the output is the pre-processed, clean data.

[0637] Step 5:

[0638] The server analyzes the preprocessed data using natural language analysis techniques (e.g., morphological analysis, topic modeling, clustering). First, morphological analysis is performed to tokenize the text. Next, Latent Dirichlet Allocation (LDA) is used to identify major topics. Next, k-means clustering is used to group related topics. The input is the preprocessed data, and the output is a list of topics and clusters resulting from the analysis.

[0639] Step 6:

[0640] The server identifies frequently occurring keywords and phrases based on the results of natural language analysis and extracts major trends and buzzwords. In this step, important keywords are extracted and listed based on the results of topic modeling and clustering. The input is the analysis results, and the output is a list of extracted major trends and buzzwords.

[0641] Step 7:

[0642] The server uses an emotion analysis engine (e.g., VADemotion) to analyze the user's input and recognize their emotions. Specifically, when a user enters a phrase that expresses an emotion, such as "surprised," the server identifies that emotion and generates emotion data as the analysis result. The input is text related to the user's emotional mood, and the output is the recognized emotion data.

[0643] Step 8:

[0644] The server takes into account the results of the sentiment analysis and generates a summary for the user. Specifically, it formats the summary into something like, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML. The input is the extracted trends, buzzwords, and sentiment data, and the output is the formatted summary.

[0645] Step 9:

[0646] The server sends the generated summary to the terminal. Specifically, the server sends summary data to the user's terminal using HTTP or the WebSocket protocol. The input is the summary in HTML format, and the output is the summary data received by the terminal.

[0647] Step 10:

[0648] The device parses the received summary and displays it to the user via a browser or dedicated application. Specifically, the device renders the received data in HTML format and displays it to the user. Through this interface, the user can check trends and buzzwords in real time. The user can also input new topics or keywords and request analysis again. The input is the received summary data, and the output is the information displayed to the user.

[0649] (Application example 2)

[0650] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0651] In today's information society, it is difficult for users to efficiently gather and understand the latest trend information. In particular, there is a need for a method to extract appropriate information from a wide range of information sources, such as social media and news sites, and provide it according to the user's interests. However, doing this manually takes a lot of time and effort. In addition, a system is needed that can provide more useful information by taking the user's emotions into consideration rather than simply collecting information.

[0652] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for performing sentiment analysis based on the analyzed trends and buzzwords, means for providing the user with a summary generated in consideration of the sentiment analysis, and means for displaying the generated summary on the user's terminal. This allows the user to efficiently obtain the latest trend information, and because the information is based on the user's emotions, more useful and comprehensive information can be provided.

[0653] "Natural language analysis technology" is a technology that uses computers to analyze text data and understand language structure.

[0654] "Noise" refers to unnecessary data or information that can cause inaccurate analysis results.

[0655] "Topic modeling" is a technique for extracting themes and themes from text data, and is a technology that analyzes data to identify potential topics.

[0656] A "clustering algorithm" is a mathematical technique for classifying data into groups, grouping related data points together.

[0657] "Sentiment analysis" is a technology that automatically extracts and analyzes user emotions and opinions from text data.

[0658] A "summary" summarizes the analyzed information and summarizes the main points.

[0659] "User terminal" refers to a communication device, such as a smartphone, tablet, or PC, that a user uses to obtain information.

[0660] This invention provides a system that efficiently collects real-time trend information based on keywords specified by a user, generates summaries taking into account sentiment analysis, and displays the summaries on the user's terminal. Specific embodiments of the system are described below.

[0661] System Configuration

[0662] This system is configured using the following hardware and software:

[0663] Hardware: Servers, user devices (smartphones, tablets, PCs)

[0664] Software: APIs (news site APIs, social media APIs), databases (MySQL, MongoDB), programming languages ​​(Python, JavaScript), NLP libraries (spaCy, NLTK), cloud services (AWS, Google Cloud)

[0665] Data collection

[0666] The server uses APIs of news sites and social media to collect related data based on keywords entered by the user on the device. This data includes text, images, videos, etc. For example, if a user enters "latest AI technology trends," the server retrieves the latest posts and articles related to "AI technology."

[0667] Data Preprocessing

[0668] The collected data is pre-processed to remove noise. Specifically, filtering algorithms are used to remove advertisements and spam, and text normalization is performed (for example, converting uppercase letters to lowercase and removing special characters). Duplicate data is also removed to improve the accuracy of the analysis.

[0669] Natural Language Analysis and Trend Extraction

[0670] After preprocessing, the data is tokenized using natural language analysis, topic modeling (e.g., Latent Dirichlet Allocation) is used to identify major topics, and then a clustering algorithm (e.g., k-means clustering) is used to group related topics and extract trends and buzzwords.

[0671] Emotion analysis

[0672] Based on the extracted trends and buzzwords, an emotion engine is used to analyze the user's emotions, thereby determining the emotions associated with the keywords entered by the user.

[0673] Summary generation and display

[0674] Taking into account the results of sentiment analysis, a summary optimized for the user is generated. For example, it may be formatted as, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." The generated summary is formatted in HTML and sent to the user's device. Users can check trend information that changes in real time and request reanalysis by entering new topics or keywords.

[0675] Specific examples

[0676] For example, if a user types "latest AI technology trends" into their device, the server will instantly collect relevant data from social media and news sites, perform sentiment analysis through the steps described above, and generate an optimal summary. Through this series of processes, users can efficiently obtain the latest trend information.

[0677] Prompt Sentence Examples

[0678] Collect data on the latest AI technology trends, perform sentiment analysis, and generate summaries based on user sentiment.

[0679] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0680] Step 1:

[0681] The user enters a trend keyword into the device. This entered keyword is sent to the server. For example, the user enters "latest AI technology trends." This keyword will become the basis for data collection in the next step.

[0682] Step 2:

[0683] The server collects related data using news site APIs and social media APIs based on the received keywords. Specifically, it retrieves posts and articles containing the user-entered "latest AI technology trends" via the API and stores the data in a database. The input is the keywords, and the output is the collected text data and metadata.

[0684] Step 3:

[0685] The server performs preprocessing to remove noise from the data stored in the database. Specifically, it uses filtering algorithms to remove advertisements and spam, and normalizes the text. At this stage, uppercase and lowercase letters are converted, special characters are removed, and duplicate data is deleted. The input is the collected text data, and the output is the preprocessed, clean text data.

[0686] Step 4:

[0687] The server tokenizes the preprocessed data using natural language processing (NLP) techniques, identifies major topics using topic modeling (e.g., Latent Dirichlet Allocation), and then groups related topics using a clustering algorithm (e.g., k-means clustering). The input is the preprocessed, clean text data, and the output is data for each topic and cluster.

[0688] Step 5:

[0689] Based on the topic and cluster information, the server analyzes the data using a sentiment analysis engine, which identifies user sentiment toward each topic or buzzword. Specifically, the server analyzes the text data and classifies it into sentiment categories such as positive, negative, and neutral. The input is data for each topic and cluster, and the output is the sentiment analysis results.

[0690] Step 6:

[0691] Based on the results of the sentiment analysis, the server generates a summary optimized for the user. For example, if a user is interested in "the latest AI technology trends," the server generates a summary such as, "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input is the sentiment analysis results and topic information, and the output is a formatted summary.

[0692] Step 7:

[0693] The server sends the generated summary to the user's device, which then displays the summary. The user can check trend information changing in real time and can also input new keywords to request another analysis. The input is the generated summary, and the output is the summary displayed on the user's device.

[0694] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0695] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0696] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0697] [Third embodiment]

[0698] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0699] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0700] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0701] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0702] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0703] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0704] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0705] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0706] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0707] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0708] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0709] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0710] This invention is a data analysis system that can grasp trends and buzzwords in real time, and performs automatic data collection and analysis to extract information from news and social media. Each element that makes up this system is explained below.

[0711] Data collection

[0712] The server collects data using the APIs of multiple social media platforms and news sites based on keywords entered by the user on their device. For example, it uses the Twitter API to obtain the latest tweets about "AI technology." It also uses the Google News API to collect related news articles. The collected data is stored in the server's database.

[0713] Data Preprocessing

[0714] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, etc., and removes unnecessary duplicate data.

[0715] Natural language analysis (NLP) technology

[0716] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation - LDA) is then used to identify major topics. After this, a clustering algorithm (e.g., k-means clustering) is used to group related topics.

[0717] Trend and buzzword extraction

[0718] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[0719] Summary Generation

[0720] Based on the extracted trends and buzzwords, the server generates an easy-to-understand summary for the user. For example, a summary like "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." This summary is formatted in HTML.

[0721] View Summary

[0722] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[0723] Specific examples

[0724] When a user types "latest AI technology trends" into their device, the server collects related data using the Twitter API and Google News API. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. A summary generated based on the extraction results is sent to the user's device and displayed. The user can view this information in real time and continue the analysis by entering new topics and keywords.

[0725] In this way, the system of the present invention utilizes natural language analysis technology to extract, analyze, and display important trends and buzzwords from vast amounts of data in real time, thereby supporting users' knowledge gathering and decision-making.

[0726] The processing flow will be explained below.

[0727] Step 1:

[0728] User enters topic or keyword

[0729] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[0730] Step 2:

[0731] Server collects data from social media and news sites

[0732] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and stores the collected data in a local database.

[0733] Step 3:

[0734] The server preprocesses the data

[0735] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[0736] Step 4:

[0737] The server performs natural language processing (NLP)

[0738] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[0739] Step 5:

[0740] Trend and buzzword extraction

[0741] The server identifies frequently occurring keywords and phrases from the NLP analysis results, and lists major trends and buzzwords based on the results of topic modeling and clustering.

[0742] Step 6:

[0743] The server generates the summary

[0744] The server generates a summary for the user based on the extracted trends and buzzwords. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular."

[0745] Step 7:

[0746] The server sends the summary to the device

[0747] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[0748] Step 8:

[0749] The device displays a summary to the user

[0750] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[0751] Example 1

[0752] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0753] It solves the difficulty of quickly and effectively identifying trends and buzzwords from vast amounts of data and providing information that interests users in real time.

[0754] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0755] In this invention, the server includes means for collecting data based on keywords specified by a user, means for removing noise from the collected data, means for preprocessing the collected data, means for analyzing the preprocessed data using natural language analysis techniques, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for generating summaries based on the extracted trends and buzzwords, and means for displaying the generated summaries on the user's terminal, thereby enabling the latest information of interest to the user to be quickly and efficiently identified and displayed.

[0756] "Means for collecting data based on user-specified keywords" is a function that allows a user to input a specific keyword of interest and obtain information related to that keyword from multiple sources.

[0757] "Means for removing noise from collected data" refers to a processing function for removing unnecessary information, advertisements, and spam messages from collected data and improving the quality of the data.

[0758] "Means for preprocessing collected data" refers to a function for formatting collected data into a format that is easy to analyze, such as converting uppercase letters to lowercase, removing special characters, and deleting unnecessary duplicate data.

[0759] "Means for analyzing preprocessed data using natural language processing technology" refers to a function that utilizes natural language processing technology to further analyze preprocessed data and divide it into meaningful units. Specifically, this includes morphological analysis and topic modeling.

[0760] "Means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms" refers to the function of identifying important topics and frequently occurring keywords and phrases from collected and preprocessed data and grouping them. Latent Dirichlet Allocation is used for topic modeling, and the k-means algorithm is used for clustering.

[0761] "Means for generating summaries based on extracted trends and buzzwords" is a function for generating easy-to-understand summaries for users based on important information identified from the analyzed data.

[0762] The "means for displaying the generated summary on the user's terminal" is a function for sending the summary information generated by the server to the user's terminal and displaying it to the user via a browser or a dedicated application.

[0763] MODE FOR CARRYING OUT THE INVENTION

[0764] This invention relates to a data analysis system that extracts and displays in real time the latest trends and buzzwords that users are interested in. This system has the ability to automatically collect and analyze data and obtain information related to specific keywords from multiple sources, such as social media and news sites.

[0765] Data collection

[0766] The server collects data using APIs of social media and news sites based on keywords entered by the user on the device. For example, the Twitter API or Google News API can be used. A specific example is collecting the latest tweets and news articles related to "AI technology." The collected data is stored in the server's database.

[0767] Data Preprocessing

[0768] The collected data is pre-processed by the server. Filtering algorithms are used to remove advertisements and spam, and text data is normalized. This includes converting uppercase to lowercase, removing special characters, and deleting unnecessary duplicate data. The pre-processed data is then stored in a clean database.

[0769] Natural language analysis (NLP) technology

[0770] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Morphological analysis is performed to tokenize the text. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. Furthermore, the k-means clustering algorithm is used to group related topics to extract important trends and buzzwords.

[0771] Summary Generation

[0772] The server generates an easy-to-understand summary for users based on the extracted trends and buzzwords. For example, a summary like "Natural language processing and machine learning are important in the latest AI technology trends. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML and stored in a database.

[0773] View Summary

[0774] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[0775] Specific examples

[0776] When a user types "latest AI technology trends" into their device, the server will collect relevant data using the Twitter API and Google News API. This data will undergo preprocessing, morphological analysis, and topic modeling to extract key topics and related trends. Finally, a summary based on the extracted results will be sent to the user's device and displayed. The following is an example of a prompt:

[0777] "Collect relevant data from Twitter and Google News about the latest AI technology trends, extract key trends and buzzwords, and generate summaries."

[0778] As a result, the system of the present invention can quickly and efficiently identify and display the latest information that is of interest to the user.

[0779] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0780] Step 1:

[0781] The user inputs keywords of interest through the terminal, for example, "latest AI technology trends." This input data is sent from the terminal to the server. The input of this process is the keyword entered by the user, and the output is the keyword being sent to the server.

[0782] Step 2:

[0783] The server receives the keywords entered by the user and starts collecting data based on them. Specifically, it calls the Twitter API and Google News API to retrieve the latest tweets and news articles related to "AI technology." The input for this data collection is the keywords entered by the user, and the output is data obtained from social media and news sites.

[0784] Step 3:

[0785] The server pre-processes the collected data, which includes filtering to remove advertisements and spam, normalizing the text, converting uppercase to lowercase, removing special characters, and removing unnecessary duplicate data. The input to this pre-processing is the raw collected data, and the output is the cleaned and formatted data.

[0786] Step 4:

[0787] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Specifically, morphological analysis is performed and the text is tokenized. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. The topics are then grouped using the k-means clustering algorithm. The input to this analysis is the preprocessed data, and the output is a list of topics and related trends.

[0788] Step 5:

[0789] The server extracts key trends and buzzwords based on the results of natural language analysis. Specifically, it identifies frequently occurring keywords and phrases and lists them using the results of topic modeling and clustering. The input for this trend extraction is the analysis results, and the output is a list of frequently occurring keywords and phrases.

[0790] Step 6:

[0791] The server generates a summary for the user based on the extracted trends and buzzwords. A specific example is a summary such as, "The latest AI technology trends are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input for this summary generation are the extracted trends and buzzwords, and the output is the generated summary.

[0792] Step 7:

[0793] The generated summary is sent from the server to the terminal. The terminal receives this summary, parses it, and displays it to the user via a browser or a dedicated application. The user can view the displayed summary and enter new keywords to continue the analysis. The input of this summary display is the generated summary, and the output is the summary displayed on the user's screen.

[0794] (Application example 1)

[0795] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0796] The explosive growth of information in today's world has made it difficult for users to effectively collect and utilize the latest trending information that matches their interests. Furthermore, information collected using conventional natural language analysis systems and topic modeling techniques often contains noise and cannot fully meet user needs. Furthermore, there is a lack of mechanisms for effectively utilizing user feedback to improve the accuracy of recommendation content. To solve these problems, a system that integrates more advanced data analysis technology and user feedback is needed.

[0797] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0798] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data based on keywords specified by the user, means for displaying the generated summary on the user's terminal, and means for collecting user feedback to improve the accuracy of recommendation content, thereby enabling users to effectively check the latest information that matches their interests and improving the accuracy of the system's recommendations through feedback.

[0799] "Natural language analysis technology" is a technology for analyzing collected text data and understanding its meaning and structure.

[0800] "Noise" is unnecessary data or information that reduces the accuracy of data analysis.

[0801] "Topic modeling" is a technique for automatically extracting latent topics from collected text data.

[0802] A "clustering algorithm" is an algorithm for dividing data into groups and aggregating highly related data.

[0803] "Latent Dirichlet Allocation" is a topic modeling technique for estimating topic distribution within documents.

[0804] The "k-means algorithm" is a clustering technique for dividing data into k clusters.

[0805] A "summary" is information that extracts the main points from a large amount of information and summarizes them concisely.

[0806] "Feedback" refers to opinions and evaluations provided by users, and is information that is useful for improving the system.

[0807] "Recommendation" is a function that recommends optimal information and content based on the user's interests and preferences.

[0808] A "user terminal" is a device that is directly operated by a user, such as a smartphone or computer.

[0809] This invention is a content recommendation system that provides users with the latest trending information in real time. This system utilizes natural language analysis technology, data preprocessing technology, topic modeling technology, and clustering algorithms. Detailed embodiments of this system are described below.

[0810] Hardware and Software Used

[0811] This system uses the following hardware and software:

[0812] Server: Data collection, preprocessing, analysis, and recommendation processing (Flask, MongoDB)

[0813] Device: Displaying information to users and collecting feedback (smartphone app: React Native, Redux)

[0814] Data collection API: Twitter API, Google News API

[0815] Data preprocessing libraries: pandas, NLTK, BeautifulSoup

[0816] Natural language analysis libraries: spaCy, gensim (LDA), scikit-learn (k-means)

[0817] Data collection

[0818] The server uses the Twitter API or Google News API to collect relevant tweets and news articles based on keywords specified by the user. For example, if the keyword "latest AI technology trends" is entered, the server will collect tweets and news articles related to this keyword.

[0819] Data Preprocessing

[0820] The collected data is preprocessed on the server. During this process, noise is removed from the text data and only the necessary data is extracted. Specific preprocessing steps include removing URLs, removing special characters, normalizing case, and removing unnecessary duplicate data.

[0821] natural language analysis

[0822] Once the data has been preprocessed, the server performs natural language analysis, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[0823] Trend extraction and summary generation

[0824] Based on the analysis, the server identifies frequently occurring keywords and phrases and extracts key trends, which allows it to identify the latest information relevant to the user's interests. It then generates a summary based on the extracted trend information and formats it in HTML.

[0825] View summary and collect feedback

[0826] The generated summary is sent from the server to the user's device. The device parses the summary and displays it to the user via a browser or dedicated application. The user can review the summary and provide feedback. This feedback is sent back to the server and used to improve the accuracy of the recommendations.

[0827] Examples of concrete examples and prompts

[0828] For example, if a user types "latest AI technology trends" into their device, the server will collect data based on the specified keywords and perform preprocessing, natural language analysis, and trend extraction. The resulting summary will be sent to the user's device and displayed in real time. The user can then use this information to enter new keywords and continue with more detailed analysis.

[0829] An example of a prompt for a generative AI model is:

[0830] "Collect news and tweets about the latest AI technology trends and identify key topics."

[0831] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0832] Step 1:

[0833] When a user inputs a keyword into the device, for example, "latest AI technology trends," the device sends this keyword to the server.

[0834] Input: User-specified keywords (e.g., "latest AI technology trends")

[0835] Output: The keyword is sent to the server

[0836] Step 2:

[0837] The server collects data based on the received keywords using the Twitter API and Google News API, calling each API to retrieve related tweets and news articles.

[0838] Input:keyword

[0839] Output: Collected tweets and news articles

[0840] Step 3:

[0841] The server preprocesses the collected data by removing noise from the text data and extracting only the necessary data, including removing URLs, special characters, case normalization, and removing unnecessary duplicate data.

[0842] Input: Collected tweets and news articles

[0843] Output: Preprocessed text data

[0844] Step 4:

[0845] The server performs natural language analysis on the preprocessed data, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[0846] Input: Preprocessed text data

[0847] Output: Topics and clustering results

[0848] Step 5:

[0849] The server uses the results of the analysis to identify frequently occurring keywords and phrases, extract key trends, and generate summaries based on this information, formatting them in HTML.

[0850] Input: Topics and clustering results

[0851] Output: Generated summary

[0852] Step 6:

[0853] The generated summary is sent from the server to the user's device, which parses it and displays it to the user via a browser or dedicated application.

[0854] Input: Generated summary

[0855] Output: A summary displayed on the terminal

[0856] Step 7:

[0857] The user reviews the displayed summary and provides feedback, which the device sends to the server.

[0858] Input: User feedback

[0859] Output: Feedback sent to the server

[0860] Step 8:

[0861] The server analyzes the received feedback to improve the accuracy of the recommendations, and appropriate trend information based on the feedback is reflected in future recommendations.

[0862] Input: User feedback

[0863] Output: Improved recommendation content

[0864] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0865] This invention is a data analysis system that can grasp trends and buzzwords in real time. It combines an emotion engine with a system that automatically collects and analyzes data and extracts information from news and social media. Each element of this system is explained below.

[0866] Data collection

[0867] Based on keywords entered by the user into the device, the server collects data using the APIs of multiple social media platforms and news sites. For example, the server may use a social media API to obtain the latest posts about "AI technology" and a news site API to collect related articles. The collected data is then stored in the server's database.

[0868] Data Preprocessing

[0869] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, and removes duplicate data.

[0870] Natural language analysis (NLP) technology

[0871] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., LDA) is then used to identify major topics. Related topics are then grouped together using a clustering algorithm (e.g., k-means clustering).

[0872] Trend and buzzword extraction

[0873] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[0874] Emotion engine integration

[0875] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion such as "surprised," the emotion engine will recognize that emotion and provide information based on that emotion.

[0876] Summary Generation

[0877] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[0878] Sending and viewing summaries

[0879] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time, enter new topics or keywords as needed, and request another analysis.

[0880] Specific examples

[0881] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary provided. Finally, the generated summary is sent to the user's device and can be viewed in real time. This process allows users to obtain more comprehensive information quickly.

[0882] The processing flow will be explained below.

[0883] Step 1:

[0884] User enters topic or keyword

[0885] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[0886] Step 2:

[0887] The server analyzes the user's input and recognizes their emotions.

[0888] The server analyzes the received input using an emotion engine to recognize the user's emotions (e.g., "surprise," "interest," "concern," etc.). The analysis results are used for subsequent data analysis and summary generation.

[0889] Step 3:

[0890] Server collects data from social media and news sites

[0891] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and the collected data is stored in the server's local database.

[0892] Step 4:

[0893] The server preprocesses the data

[0894] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[0895] Step 5:

[0896] The server performs natural language processing (NLP)

[0897] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[0898] Step 6:

[0899] Trend and buzzword extraction

[0900] The server identifies frequently occurring keywords and phrases from the NLP analysis results. Based on the results of topic modeling and clustering, it lists major trends and buzzwords. The results of the sentiment engine are also taken into account at this stage.

[0901] Step 7:

[0902] Summary Generation

[0903] The server generates a summary for the user based on the extracted trends and buzzwords and the results of the emotion engine analysis. For example, it might say, "The latest trends in AI technology that you are interested in are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are attracting attention." The generated summary is formatted in HTML.

[0904] Step 8:

[0905] Sending a summary

[0906] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[0907] Step 9:

[0908] The device displays a summary to the user

[0909] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[0910] Specific examples

[0911] 1. User Input

[0912] The user types "latest AI technology trends" into the input field on their device.

[0913] The terminal sends the entered keyword to the server.

[0914] 2. Emotion analysis

[0915] The server analyzes the user's input using an emotion engine and recognizes their "interests and concerns."

[0916] 3. Data Collection

[0917] The server uses the Twitter API and Google News API to collect the latest data related to "AI technology."

[0918] 4. Data Preprocessing

[0919] The server uses filtering algorithms to remove advertisements and spam and normalize the text.

[0920] 5. Natural language analysis

[0921] The server performs morphological analysis and performs topic modeling and clustering.

[0922] 6. Trend Extraction

[0923] The server extracts frequently appearing keywords (e.g., "GPT-4," "BERT," and "deep learning") from the analysis results.

[0924] 7. Summary Generation

[0925] The server takes into account the analysis results of the emotion engine and generates a summary that reads, "The latest trends in AI technology are natural language processing and machine learning. GPT-4 and BERT in particular are attracting attention."

[0926] 8. Summary Transmission

[0927] The server sends the generated summary to the terminal.

[0928] 9. Summary View

[0929] The user views the summary received on the device and requests further analysis using new topics or keywords.

[0930] This process allows users to receive the latest trend information quickly, emotionally, and appropriately, helping them make decisions.

[0931] Example 2

[0932] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0933] Conventional data analysis systems have difficulty grasping trends and buzzwords that correspond to user interests in real time. Furthermore, because sentiment analysis is not integrated, they are unable to provide information that corresponds to the user's emotions and are unable to generate appropriate summaries. As a result, users are unable to obtain information quickly and accurately, resulting in lower search accuracy and lower satisfaction.

[0934] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0935] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data from multiple sources based on user-specified keywords, means for integrating a sentiment analysis engine to understand user sentiment and adjust the generated summary based on the sentiment, and means for transmitting the generated summary to the user's terminal and displaying it. This makes it possible to provide the latest information according to the user's interests in real time and generate appropriate summaries that take the user's sentiment into consideration.

[0936] "Natural language analysis technology" is a technology that analyzes text data and grasps meaning and structure through morphological analysis, topic modeling, clustering, etc.

[0937] "Noise" refers to unnecessary information or data in data analysis, including advertisements, spam, and duplicate data.

[0938] "Topic modeling" is a method for extracting latent topics from large amounts of text data, and a representative algorithm is Latent Dirichlet Allocation (LDA).

[0939] A "clustering algorithm" is a method for grouping data based on similar characteristics, and a representative algorithm is the k-means algorithm.

[0940] "Trends" refer to subjects or keywords that are frequently mentioned during a particular period of time.

[0941] A "buzzword" is a keyword or phrase that rapidly gains attention in a short period of time.

[0942] "User-specified keywords" are words or phrases that a user inputs into the system to be used as search and analysis criteria.

[0943] An "emotion analysis engine" is a system that analyzes emotions contained in text data and recognizes emotional states such as positive, negative, and surprise.

[0944] A "summary" is summary information generated based on the analysis results, and is information in a format that is organized so that the user can easily understand it.

[0945] A "terminal" is a hardware device that allows a user to communicate with the system through an interface, and includes a personal computer, a smartphone, etc.

[0946] The present invention is a data analysis system that can grasp trends and buzzwords in real time, and combines an emotion engine with a system in which a server automatically collects and analyzes data and extracts information from news and social media. Details of an embodiment of this system are provided below.

[0947] Data collection

[0948] The user enters a keyword into the device. For example, they might enter "latest AI technology trends." Based on this keyword, the server collects data using multiple social media APIs (e.g., Twitter API) and news site APIs (e.g., News API). In doing so, the server collects posts and articles related to the specified keyword. The collected data is stored in the server's database (e.g., MongoDB or MySQL).

[0949] Data Preprocessing

[0950] The collected data is pre-processed to remove noise: the server uses filtering algorithms (e.g., regular expression filters) to remove advertisements and spam, and normalizes the text, converting it to lowercase, removing special characters, and deleting duplicate data.

[0951] Natural language analysis (NLP)

[0952] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation, LDA) is then used to identify major topics, followed by a clustering algorithm (e.g., k-means clustering) to group related topics.

[0953] Trend and buzzword extraction

[0954] Based on the results of natural language analysis, the server identifies frequently occurring keywords and phrases. Based on the results of topic modeling and clustering, it lists major trends and buzzwords, thereby identifying the latest information that matches the user's interests.

[0955] Emotion engine integration

[0956] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion, such as "surprised," the emotion engine (e.g., VADemotion) will recognize that emotion and adjust the information provided accordingly.

[0957] Summary Generation

[0958] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[0959] Sending and viewing summaries

[0960] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time. If necessary, they can also input new topics or keywords and request another analysis.

[0961] Specific examples

[0962] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary content provided. Finally, the generated summary is sent to the user's device and can be viewed in real time.

[0963] Prompt Sentence Examples

[0964] "What are the latest trends in AI technology?"

[0965] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0966] Step 1:

[0967] The user inputs a keyword into the device. An example input is "latest AI technology trends." This input keyword becomes the basis for the subsequent data collection process. The input keyword is sent to the server.

[0968] Step 2:

[0969] Based on the keywords received from the user, the server calls social media APIs (e.g., Twitter API) and news site APIs (e.g., News API) to collect related data. Specifically, the server sends a query using the keywords to each API, and the retrieved data is returned to the server in JSON format. This data includes related posts and articles.

[0970] Step 3:

[0971] The server stores the collected data in a database (e.g., MongoDB or MySQL). A database is used to efficiently organize the data and store it in a searchable format. When storing, the JSON data is split into fields based on the appropriate schema and inserted.

[0972] Step 4:

[0973] The server performs pre-processing on the collected data. In this step, regular expression filters are used to remove advertisements and spam, and text normalization is performed. Specific operations include converting uppercase to lowercase, removing special characters, and deleting duplicates. The input is raw data retrieved from the database, and the output is the pre-processed, clean data.

[0974] Step 5:

[0975] The server analyzes the preprocessed data using natural language analysis techniques (e.g., morphological analysis, topic modeling, clustering). First, morphological analysis is performed to tokenize the text. Next, Latent Dirichlet Allocation (LDA) is used to identify major topics. Next, k-means clustering is used to group related topics. The input is the preprocessed data, and the output is a list of topics and clusters resulting from the analysis.

[0976] Step 6:

[0977] The server identifies frequently occurring keywords and phrases based on the results of natural language analysis and extracts major trends and buzzwords. In this step, important keywords are extracted and listed based on the results of topic modeling and clustering. The input is the analysis results, and the output is a list of extracted major trends and buzzwords.

[0978] Step 7:

[0979] The server uses an emotion analysis engine (e.g., VADemotion) to analyze the user's input and recognize their emotions. Specifically, when a user enters a phrase that expresses an emotion, such as "surprised," the server identifies that emotion and generates emotion data as the analysis result. The input is text related to the user's emotional mood, and the output is the recognized emotion data.

[0980] Step 8:

[0981] The server takes into account the results of the sentiment analysis and generates a summary for the user. Specifically, it formats the summary into something like, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML. The input is the extracted trends, buzzwords, and sentiment data, and the output is the formatted summary.

[0982] Step 9:

[0983] The server sends the generated summary to the terminal. Specifically, the server sends summary data to the user's terminal using HTTP or the WebSocket protocol. The input is the summary in HTML format, and the output is the summary data received by the terminal.

[0984] Step 10:

[0985] The device parses the received summary and displays it to the user via a browser or dedicated application. Specifically, the device renders the received data in HTML format and displays it to the user. Through this interface, the user can check trends and buzzwords in real time. The user can also input new topics or keywords and request analysis again. The input is the received summary data, and the output is the information displayed to the user.

[0986] (Application example 2)

[0987] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0988] In today's information society, it is difficult for users to efficiently gather and understand the latest trend information. In particular, there is a need for a method to extract appropriate information from a wide range of information sources, such as social media and news sites, and provide it according to the user's interests. However, doing this manually takes a lot of time and effort. In addition, a system is needed that can provide more useful information by taking the user's emotions into consideration rather than simply collecting information.

[0989] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for performing sentiment analysis based on the analyzed trends and buzzwords, means for providing the user with a summary generated in consideration of the sentiment analysis, and means for displaying the generated summary on the user's terminal. This allows the user to efficiently obtain the latest trend information, and because the information is based on the user's emotions, more useful and comprehensive information can be provided.

[0990] "Natural language analysis technology" is a technology that uses computers to analyze text data and understand language structure.

[0991] "Noise" refers to unnecessary data or information that can cause inaccurate analysis results.

[0992] "Topic modeling" is a technique for extracting themes and themes from text data, and is a technology that analyzes data to identify potential topics.

[0993] A "clustering algorithm" is a mathematical technique for classifying data into groups, grouping related data points together.

[0994] "Sentiment analysis" is a technology that automatically extracts and analyzes user emotions and opinions from text data.

[0995] A "summary" summarizes the analyzed information and summarizes the main points.

[0996] "User terminal" refers to a communication device, such as a smartphone, tablet, or PC, that a user uses to obtain information.

[0997] This invention provides a system that efficiently collects real-time trend information based on keywords specified by a user, generates summaries taking into account sentiment analysis, and displays the summaries on the user's terminal. Specific embodiments of the system are described below.

[0998] System Configuration

[0999] This system is configured using the following hardware and software:

[1000] Hardware: Servers, user devices (smartphones, tablets, PCs)

[1001] Software: APIs (news site APIs, social media APIs), databases (MySQL, MongoDB), programming languages ​​(Python, JavaScript), NLP libraries (spaCy, NLTK), cloud services (AWS, Google Cloud)

[1002] Data collection

[1003] The server uses APIs of news sites and social media to collect related data based on keywords entered by the user on the device. This data includes text, images, videos, etc. For example, if a user enters "latest AI technology trends," the server retrieves the latest posts and articles related to "AI technology."

[1004] Data Preprocessing

[1005] The collected data is pre-processed to remove noise. Specifically, filtering algorithms are used to remove advertisements and spam, and text normalization is performed (for example, converting uppercase letters to lowercase and removing special characters). Duplicate data is also removed to improve the accuracy of the analysis.

[1006] Natural Language Analysis and Trend Extraction

[1007] After preprocessing, the data is tokenized using natural language analysis, topic modeling (e.g., Latent Dirichlet Allocation) is used to identify major topics, and then a clustering algorithm (e.g., k-means clustering) is used to group related topics and extract trends and buzzwords.

[1008] Emotion analysis

[1009] Based on the extracted trends and buzzwords, an emotion engine is used to analyze the user's emotions, thereby determining the emotions associated with the keywords entered by the user.

[1010] Summary generation and display

[1011] Taking into account the results of sentiment analysis, a summary optimized for the user is generated. For example, it may be formatted as, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." The generated summary is formatted in HTML and sent to the user's device. Users can check trend information that changes in real time and request reanalysis by entering new topics or keywords.

[1012] Specific examples

[1013] For example, if a user types "latest AI technology trends" into their device, the server will instantly collect relevant data from social media and news sites, perform sentiment analysis through the steps described above, and generate an optimal summary. Through this series of processes, users can efficiently obtain the latest trend information.

[1014] Prompt Sentence Examples

[1015] Collect data on the latest AI technology trends, perform sentiment analysis, and generate summaries based on user sentiment.

[1016] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1017] Step 1:

[1018] The user enters a trend keyword into the device. This entered keyword is sent to the server. For example, the user enters "latest AI technology trends." This keyword will become the basis for data collection in the next step.

[1019] Step 2:

[1020] The server collects related data using news site APIs and social media APIs based on the received keywords. Specifically, it retrieves posts and articles containing the user-entered "latest AI technology trends" via the API and stores the data in a database. The input is the keywords, and the output is the collected text data and metadata.

[1021] Step 3:

[1022] The server performs preprocessing to remove noise from the data stored in the database. Specifically, it uses filtering algorithms to remove advertisements and spam, and normalizes the text. At this stage, uppercase and lowercase letters are converted, special characters are removed, and duplicate data is deleted. The input is the collected text data, and the output is the preprocessed, clean text data.

[1023] Step 4:

[1024] The server tokenizes the preprocessed data using natural language processing (NLP) techniques, identifies major topics using topic modeling (e.g., Latent Dirichlet Allocation), and then groups related topics using a clustering algorithm (e.g., k-means clustering). The input is the preprocessed, clean text data, and the output is data for each topic and cluster.

[1025] Step 5:

[1026] Based on the topic and cluster information, the server analyzes the data using a sentiment analysis engine, which identifies user sentiment toward each topic or buzzword. Specifically, the server analyzes the text data and classifies it into sentiment categories such as positive, negative, and neutral. The input is data for each topic and cluster, and the output is the sentiment analysis results.

[1027] Step 6:

[1028] Based on the results of the sentiment analysis, the server generates a summary optimized for the user. For example, if a user is interested in "the latest AI technology trends," the server generates a summary such as, "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input is the sentiment analysis results and topic information, and the output is a formatted summary.

[1029] Step 7:

[1030] The server sends the generated summary to the user's device, which then displays the summary. The user can check trend information changing in real time and can also input new keywords to request another analysis. The input is the generated summary, and the output is the summary displayed on the user's device.

[1031] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1032] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1033] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1034] [Fourth embodiment]

[1035] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1036] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1037] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1038] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1039] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1040] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1041] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1042] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1043] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1044] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1045] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1046] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1047] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1048] This invention is a data analysis system that can grasp trends and buzzwords in real time, and performs automatic data collection and analysis to extract information from news and social media. Each element that makes up this system is explained below.

[1049] Data collection

[1050] The server collects data using the APIs of multiple social media platforms and news sites based on keywords entered by the user on their device. For example, it uses the Twitter API to obtain the latest tweets about "AI technology." It also uses the Google News API to collect related news articles. The collected data is stored in the server's database.

[1051] Data Preprocessing

[1052] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, etc., and removes unnecessary duplicate data.

[1053] Natural language analysis (NLP) technology

[1054] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation - LDA) is then used to identify major topics. After this, a clustering algorithm (e.g., k-means clustering) is used to group related topics.

[1055] Trend and buzzword extraction

[1056] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[1057] Summary Generation

[1058] Based on the extracted trends and buzzwords, the server generates an easy-to-understand summary for the user. For example, a summary like "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." This summary is formatted in HTML.

[1059] View Summary

[1060] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[1061] Specific examples

[1062] When a user types "latest AI technology trends" into their device, the server collects related data using the Twitter API and Google News API. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. A summary generated based on the extraction results is sent to the user's device and displayed. The user can view this information in real time and continue the analysis by entering new topics and keywords.

[1063] In this way, the system of the present invention utilizes natural language analysis technology to extract, analyze, and display important trends and buzzwords from vast amounts of data in real time, thereby supporting users' knowledge gathering and decision-making.

[1064] The processing flow will be explained below.

[1065] Step 1:

[1066] User enters topic or keyword

[1067] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[1068] Step 2:

[1069] Server collects data from social media and news sites

[1070] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and stores the collected data in a local database.

[1071] Step 3:

[1072] The server preprocesses the data

[1073] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[1074] Step 4:

[1075] The server performs natural language processing (NLP)

[1076] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[1077] Step 5:

[1078] Trend and buzzword extraction

[1079] The server identifies frequently occurring keywords and phrases from the NLP analysis results, and lists major trends and buzzwords based on the results of topic modeling and clustering.

[1080] Step 6:

[1081] The server generates the summary

[1082] The server generates a summary for the user based on the extracted trends and buzzwords. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular."

[1083] Step 7:

[1084] The server sends the summary to the device

[1085] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[1086] Step 8:

[1087] The device displays a summary to the user

[1088] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[1089] Example 1

[1090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1091] It solves the difficulty of quickly and effectively identifying trends and buzzwords from vast amounts of data and providing information that interests users in real time.

[1092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1093] In this invention, the server includes means for collecting data based on keywords specified by a user, means for removing noise from the collected data, means for preprocessing the collected data, means for analyzing the preprocessed data using natural language analysis techniques, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for generating summaries based on the extracted trends and buzzwords, and means for displaying the generated summaries on the user's terminal, thereby enabling the latest information of interest to the user to be quickly and efficiently identified and displayed.

[1094] "Means for collecting data based on user-specified keywords" is a function that allows a user to input a specific keyword of interest and obtain information related to that keyword from multiple sources.

[1095] "Means for removing noise from collected data" refers to a processing function for removing unnecessary information, advertisements, and spam messages from collected data and improving the quality of the data.

[1096] "Means for preprocessing collected data" refers to a function for formatting collected data into a format that is easy to analyze, such as converting uppercase letters to lowercase, removing special characters, and deleting unnecessary duplicate data.

[1097] "Means for analyzing preprocessed data using natural language processing technology" refers to a function that utilizes natural language processing technology to further analyze preprocessed data and divide it into meaningful units. Specifically, this includes morphological analysis and topic modeling.

[1098] "Means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms" refers to the function of identifying important topics and frequently occurring keywords and phrases from collected and preprocessed data and grouping them. Latent Dirichlet Allocation is used for topic modeling, and the k-means algorithm is used for clustering.

[1099] "Means for generating summaries based on extracted trends and buzzwords" is a function for generating easy-to-understand summaries for users based on important information identified from the analyzed data.

[1100] The "means for displaying the generated summary on the user's terminal" is a function for sending the summary information generated by the server to the user's terminal and displaying it to the user via a browser or a dedicated application.

[1101] MODE FOR CARRYING OUT THE INVENTION

[1102] This invention relates to a data analysis system that extracts and displays in real time the latest trends and buzzwords that users are interested in. This system has the ability to automatically collect and analyze data and obtain information related to specific keywords from multiple sources, such as social media and news sites.

[1103] Data collection

[1104] The server collects data using APIs of social media and news sites based on keywords entered by the user on the device. For example, the Twitter API or Google News API can be used. A specific example is collecting the latest tweets and news articles related to "AI technology." The collected data is stored in the server's database.

[1105] Data Preprocessing

[1106] The collected data is pre-processed by the server. Filtering algorithms are used to remove advertisements and spam, and text data is normalized. This includes converting uppercase to lowercase, removing special characters, and deleting unnecessary duplicate data. The pre-processed data is then stored in a clean database.

[1107] Natural language analysis (NLP) technology

[1108] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Morphological analysis is performed to tokenize the text. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. Furthermore, the k-means clustering algorithm is used to group related topics to extract important trends and buzzwords.

[1109] Summary Generation

[1110] The server generates an easy-to-understand summary for users based on the extracted trends and buzzwords. For example, a summary like "Natural language processing and machine learning are important in the latest AI technology trends. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML and stored in a database.

[1111] View Summary

[1112] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check and utilize trends and buzzwords that change in real time.

[1113] Specific examples

[1114] When a user types "latest AI technology trends" into their device, the server will collect relevant data using the Twitter API and Google News API. This data will undergo preprocessing, morphological analysis, and topic modeling to extract key topics and related trends. Finally, a summary based on the extracted results will be sent to the user's device and displayed. The following is an example of a prompt:

[1115] "Collect relevant data from Twitter and Google News about the latest AI technology trends, extract key trends and buzzwords, and generate summaries."

[1116] As a result, the system of the present invention can quickly and efficiently identify and display the latest information that is of interest to the user.

[1117] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1118] Step 1:

[1119] The user inputs keywords of interest through the terminal, for example, "latest AI technology trends." This input data is sent from the terminal to the server. The input of this process is the keyword entered by the user, and the output is the keyword being sent to the server.

[1120] Step 2:

[1121] The server receives the keywords entered by the user and starts collecting data based on them. Specifically, it calls the Twitter API and Google News API to retrieve the latest tweets and news articles related to "AI technology." The input for this data collection is the keywords entered by the user, and the output is data obtained from social media and news sites.

[1122] Step 3:

[1123] The server pre-processes the collected data, which includes filtering to remove advertisements and spam, normalizing the text, converting uppercase to lowercase, removing special characters, and removing unnecessary duplicate data. The input to this pre-processing is the raw collected data, and the output is the cleaned and formatted data.

[1124] Step 4:

[1125] After preprocessing, the data is analyzed on the server using natural language analysis techniques. Specifically, morphological analysis is performed and the text is tokenized. Topic modeling using Latent Dirichlet Allocation (LDA) is then performed to identify major topics. The topics are then grouped using the k-means clustering algorithm. The input to this analysis is the preprocessed data, and the output is a list of topics and related trends.

[1126] Step 5:

[1127] The server extracts key trends and buzzwords based on the results of natural language analysis. Specifically, it identifies frequently occurring keywords and phrases and lists them using the results of topic modeling and clustering. The input for this trend extraction is the analysis results, and the output is a list of frequently occurring keywords and phrases.

[1128] Step 6:

[1129] The server generates a summary for the user based on the extracted trends and buzzwords. A specific example is a summary such as, "The latest AI technology trends are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input for this summary generation are the extracted trends and buzzwords, and the output is the generated summary.

[1130] Step 7:

[1131] The generated summary is sent from the server to the terminal. The terminal receives this summary, parses it, and displays it to the user via a browser or a dedicated application. The user can view the displayed summary and enter new keywords to continue the analysis. The input of this summary display is the generated summary, and the output is the summary displayed on the user's screen.

[1132] (Application example 1)

[1133] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1134] The explosive growth of information in today's world has made it difficult for users to effectively collect and utilize the latest trending information that matches their interests. Furthermore, information collected using conventional natural language analysis systems and topic modeling techniques often contains noise and cannot fully meet user needs. Furthermore, there is a lack of mechanisms for effectively utilizing user feedback to improve the accuracy of recommendation content. To solve these problems, a system that integrates more advanced data analysis technology and user feedback is needed.

[1135] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1136] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data based on keywords specified by the user, means for displaying the generated summary on the user's terminal, and means for collecting user feedback to improve the accuracy of recommendation content, thereby enabling users to effectively check the latest information that matches their interests and improving the accuracy of the system's recommendations through feedback.

[1137] "Natural language analysis technology" is a technology for analyzing collected text data and understanding its meaning and structure.

[1138] "Noise" is unnecessary data or information that reduces the accuracy of data analysis.

[1139] "Topic modeling" is a technique for automatically extracting latent topics from collected text data.

[1140] A "clustering algorithm" is an algorithm for dividing data into groups and aggregating highly related data.

[1141] "Latent Dirichlet Allocation" is a topic modeling technique for estimating topic distribution within documents.

[1142] The "k-means algorithm" is a clustering technique for dividing data into k clusters.

[1143] A "summary" is information that extracts the main points from a large amount of information and summarizes them concisely.

[1144] "Feedback" refers to opinions and evaluations provided by users, and is information that is useful for improving the system.

[1145] "Recommendation" is a function that recommends optimal information and content based on the user's interests and preferences.

[1146] A "user terminal" is a device that is directly operated by a user, such as a smartphone or computer.

[1147] This invention is a content recommendation system that provides users with the latest trending information in real time. This system utilizes natural language analysis technology, data preprocessing technology, topic modeling technology, and clustering algorithms. Detailed embodiments of this system are described below.

[1148] Hardware and Software Used

[1149] This system uses the following hardware and software:

[1150] Server: Data collection, preprocessing, analysis, and recommendation processing (Flask, MongoDB)

[1151] Device: Displaying information to users and collecting feedback (smartphone app: React Native, Redux)

[1152] Data collection API: Twitter API, Google News API

[1153] Data preprocessing libraries: pandas, NLTK, BeautifulSoup

[1154] Natural language analysis libraries: spaCy, gensim (LDA), scikit-learn (k-means)

[1155] Data collection

[1156] The server uses the Twitter API or Google News API to collect relevant tweets and news articles based on keywords specified by the user. For example, if the keyword "latest AI technology trends" is entered, the server will collect tweets and news articles related to this keyword.

[1157] Data Preprocessing

[1158] The collected data is preprocessed on the server. During this process, noise is removed from the text data and only the necessary data is extracted. Specific preprocessing steps include removing URLs, removing special characters, normalizing case, and removing unnecessary duplicate data.

[1159] natural language analysis

[1160] Once the data has been preprocessed, the server performs natural language analysis, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[1161] Trend extraction and summary generation

[1162] Based on the analysis, the server identifies frequently occurring keywords and phrases and extracts key trends, which allows it to identify the latest information relevant to the user's interests. It then generates a summary based on the extracted trend information and formats it in HTML.

[1163] View summary and collect feedback

[1164] The generated summary is sent from the server to the user's device. The device parses the summary and displays it to the user via a browser or dedicated application. The user can review the summary and provide feedback. This feedback is sent back to the server and used to improve the accuracy of the recommendations.

[1165] Examples of concrete examples and prompts

[1166] For example, if a user types "latest AI technology trends" into their device, the server will collect data based on the specified keywords and perform preprocessing, natural language analysis, and trend extraction. The resulting summary will be sent to the user's device and displayed in real time. The user can then use this information to enter new keywords and continue with more detailed analysis.

[1167] An example of a prompt for a generative AI model is:

[1168] "Collect news and tweets about the latest AI technology trends and identify key topics."

[1169] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1170] Step 1:

[1171] When a user inputs a keyword into the device, for example, "latest AI technology trends," the device sends this keyword to the server.

[1172] Input: User-specified keywords (e.g., "latest AI technology trends")

[1173] Output: The keyword is sent to the server

[1174] Step 2:

[1175] The server collects data based on the received keywords using the Twitter API and Google News API, calling each API to retrieve related tweets and news articles.

[1176] Input:keyword

[1177] Output: Collected tweets and news articles

[1178] Step 3:

[1179] The server preprocesses the collected data by removing noise from the text data and extracting only the necessary data, including removing URLs, special characters, case normalization, and removing unnecessary duplicate data.

[1180] Input: Collected tweets and news articles

[1181] Output: Preprocessed text data

[1182] Step 4:

[1183] The server performs natural language analysis on the preprocessed data, tokenizing the text and performing morphological analysis. It then uses topic modeling (Latent Dirichlet Allocation - LDA) to identify major topics and a clustering algorithm (k-means) to group related topics.

[1184] Input: Preprocessed text data

[1185] Output: Topics and clustering results

[1186] Step 5:

[1187] The server uses the results of the analysis to identify frequently occurring keywords and phrases, extract key trends, and generate summaries based on this information, formatting them in HTML.

[1188] Input: Topics and clustering results

[1189] Output: Generated summary

[1190] Step 6:

[1191] The generated summary is sent from the server to the user's device, which parses it and displays it to the user via a browser or dedicated application.

[1192] Input: Generated summary

[1193] Output: A summary displayed on the terminal

[1194] Step 7:

[1195] The user reviews the displayed summary and provides feedback, which the device sends to the server.

[1196] Input: User feedback

[1197] Output: Feedback sent to the server

[1198] Step 8:

[1199] The server analyzes the received feedback to improve the accuracy of the recommendations, and appropriate trend information based on the feedback is reflected in future recommendations.

[1200] Input: User feedback

[1201] Output: Improved recommendation content

[1202] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1203] This invention is a data analysis system that can grasp trends and buzzwords in real time. It combines an emotion engine with a system that automatically collects and analyzes data and extracts information from news and social media. Each element of this system is explained below.

[1204] Data collection

[1205] Based on keywords entered by the user into the device, the server collects data using the APIs of multiple social media platforms and news sites. For example, the server may use a social media API to obtain the latest posts about "AI technology" and a news site API to collect related articles. The collected data is then stored in the server's database.

[1206] Data Preprocessing

[1207] The collected data is pre-processed to remove noise. The server uses filtering algorithms to remove advertisements and spam, normalizes the text, converts uppercase letters to lowercase, removes special characters, and removes duplicate data.

[1208] Natural language analysis (NLP) technology

[1209] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., LDA) is then used to identify major topics. Related topics are then grouped together using a clustering algorithm (e.g., k-means clustering).

[1210] Trend and buzzword extraction

[1211] Based on natural language analysis, the server identifies frequently occurring keywords and phrases. Topic modeling and clustering results in a list of major trends and buzzwords, identifying the latest information relevant to the user's interests.

[1212] Emotion engine integration

[1213] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion such as "surprised," the emotion engine will recognize that emotion and provide information based on that emotion.

[1214] Summary Generation

[1215] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[1216] Sending and viewing summaries

[1217] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time, enter new topics or keywords as needed, and request another analysis.

[1218] Specific examples

[1219] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary provided. Finally, the generated summary is sent to the user's device and can be viewed in real time. This process allows users to obtain more comprehensive information quickly.

[1220] The processing flow will be explained below.

[1221] Step 1:

[1222] User enters topic or keyword

[1223] The user enters a topic or keyword of interest (e.g., "AI technology") into the input field of the device, and the device sends the entered keyword to the server.

[1224] Step 2:

[1225] The server analyzes the user's input and recognizes their emotions.

[1226] The server analyzes the received input using an emotion engine to recognize the user's emotions (e.g., "surprise," "interest," "concern," etc.). The analysis results are used for subsequent data analysis and summary generation.

[1227] Step 3:

[1228] Server collects data from social media and news sites

[1229] The server uses a designated API (e.g., a social media API or a news site API) to collect relevant data based on the entered keywords, and the collected data is stored in the server's local database.

[1230] Step 4:

[1231] The server preprocesses the data

[1232] The server pre-processes the collected data, filtering out advertisements and spam, cleaning the data, normalizing it by converting it to lowercase and removing special characters, and finally removing unnecessary duplicate data.

[1233] Step 5:

[1234] The server performs natural language processing (NLP)

[1235] The server performs morphological analysis on the preprocessed data to split the text into tokens, then uses topic modeling (e.g., LDA) to identify major topics, and then uses a clustering algorithm (e.g., k-means clustering) to group related topics.

[1236] Step 6:

[1237] Trend and buzzword extraction

[1238] The server identifies frequently occurring keywords and phrases from the NLP analysis results. Based on the results of topic modeling and clustering, it lists major trends and buzzwords. The results of the sentiment engine are also taken into account at this stage.

[1239] Step 7:

[1240] Summary Generation

[1241] The server generates a summary for the user based on the extracted trends and buzzwords and the results of the emotion engine analysis. For example, it might say, "The latest trends in AI technology that you are interested in are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are attracting attention." The generated summary is formatted in HTML.

[1242] Step 8:

[1243] Sending a summary

[1244] The server converts the generated summary into JSON or XML format and sends it to the terminal as an HTTP response.

[1245] Step 9:

[1246] The device displays a summary to the user

[1247] The device parses the received summary and displays it to the user via a browser or application. The user can view the summary, enter new topics or keywords as needed, and request further analysis.

[1248] Specific examples

[1249] 1. User Input

[1250] The user types "latest AI technology trends" into the input field on their device.

[1251] The terminal sends the entered keyword to the server.

[1252] 2. Emotion analysis

[1253] The server analyzes the user's input using an emotion engine and recognizes their "interests and concerns."

[1254] 3. Data Collection

[1255] The server uses the Twitter API and Google News API to collect the latest data related to "AI technology."

[1256] 4. Data Preprocessing

[1257] The server uses filtering algorithms to remove advertisements and spam and normalize the text.

[1258] 5. Natural language analysis

[1259] The server performs morphological analysis and performs topic modeling and clustering.

[1260] 6. Trend Extraction

[1261] The server extracts frequently appearing keywords (e.g., "GPT-4," "BERT," and "deep learning") from the analysis results.

[1262] 7. Summary Generation

[1263] The server takes into account the analysis results of the emotion engine and generates a summary that reads, "The latest trends in AI technology are natural language processing and machine learning. GPT-4 and BERT in particular are attracting attention."

[1264] 8. Summary Transmission

[1265] The server sends the generated summary to the terminal.

[1266] 9. Summary View

[1267] The user views the summary received on the device and requests further analysis using new topics or keywords.

[1268] This process allows users to receive the latest trend information quickly, emotionally, and appropriately, helping them make decisions.

[1269] Example 2

[1270] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1271] Conventional data analysis systems have difficulty grasping trends and buzzwords that correspond to user interests in real time. Furthermore, because sentiment analysis is not integrated, they are unable to provide information that corresponds to the user's emotions and are unable to generate appropriate summaries. As a result, users are unable to obtain information quickly and accurately, resulting in lower search accuracy and lower satisfaction.

[1272] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1273] In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for collecting data from multiple sources based on user-specified keywords, means for integrating a sentiment analysis engine to understand user sentiment and adjust the generated summary based on the sentiment, and means for transmitting the generated summary to the user's terminal and displaying it. This makes it possible to provide the latest information according to the user's interests in real time and generate appropriate summaries that take the user's sentiment into consideration.

[1274] "Natural language analysis technology" is a technology that analyzes text data and grasps meaning and structure through morphological analysis, topic modeling, clustering, etc.

[1275] "Noise" refers to unnecessary information or data in data analysis, including advertisements, spam, and duplicate data.

[1276] "Topic modeling" is a method for extracting latent topics from large amounts of text data, and a representative algorithm is Latent Dirichlet Allocation (LDA).

[1277] A "clustering algorithm" is a method for grouping data based on similar characteristics, and a representative algorithm is the k-means algorithm.

[1278] "Trends" refer to subjects or keywords that are frequently mentioned during a particular period of time.

[1279] A "buzzword" is a keyword or phrase that rapidly gains attention in a short period of time.

[1280] "User-specified keywords" are words or phrases that a user inputs into the system to be used as search and analysis criteria.

[1281] An "emotion analysis engine" is a system that analyzes emotions contained in text data and recognizes emotional states such as positive, negative, and surprise.

[1282] A "summary" is summary information generated based on the analysis results, and is information in a format that is organized so that the user can easily understand it.

[1283] A "terminal" is a hardware device that allows a user to communicate with the system through an interface, and includes a personal computer, a smartphone, etc.

[1284] The present invention is a data analysis system that can grasp trends and buzzwords in real time, and combines an emotion engine with a system in which a server automatically collects and analyzes data and extracts information from news and social media. Details of an embodiment of this system are provided below.

[1285] Data collection

[1286] The user enters a keyword into the device. For example, they might enter "latest AI technology trends." Based on this keyword, the server collects data using multiple social media APIs (e.g., Twitter API) and news site APIs (e.g., News API). In doing so, the server collects posts and articles related to the specified keyword. The collected data is stored in the server's database (e.g., MongoDB or MySQL).

[1287] Data Preprocessing

[1288] The collected data is pre-processed to remove noise: the server uses filtering algorithms (e.g., regular expression filters) to remove advertisements and spam, and normalizes the text, converting it to lowercase, removing special characters, and deleting duplicate data.

[1289] Natural language analysis (NLP)

[1290] Once the data has been preprocessed, it is analyzed using natural language analysis techniques. The server performs morphological analysis and tokenizes the text. Topic modeling (e.g., Latent Dirichlet Allocation, LDA) is then used to identify major topics, followed by a clustering algorithm (e.g., k-means clustering) to group related topics.

[1291] Trend and buzzword extraction

[1292] Based on the results of natural language analysis, the server identifies frequently occurring keywords and phrases. Based on the results of topic modeling and clustering, it lists major trends and buzzwords, thereby identifying the latest information that matches the user's interests.

[1293] Emotion engine integration

[1294] The system is equipped with an emotion engine that analyzes user input and recognizes emotions. For example, if a user enters a phrase that indicates an emotion, such as "surprised," the emotion engine (e.g., VADemotion) will recognize that emotion and adjust the information provided accordingly.

[1295] Summary Generation

[1296] Taking into account the analysis results of the emotion engine, the server generates a summary for the user. The generated summary is formatted, for example, as follows: "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is then formatted in HTML.

[1297] Sending and viewing summaries

[1298] The generated summary is sent from the server to the device, which parses it and displays it to the user via a browser or dedicated application. Through this interface, the user can check trends and buzzwords that change in real time. If necessary, they can also input new topics or keywords and request another analysis.

[1299] Specific examples

[1300] When a user types "latest AI technology trends" into their device, the server collects related data using social media APIs and news site APIs. After preprocessing, it performs morphological analysis and topic modeling to extract key topics and related trends. An emotion engine recognizes the user's emotions and optimizes the information and summary content provided. Finally, the generated summary is sent to the user's device and can be viewed in real time.

[1301] Prompt Sentence Examples

[1302] "What are the latest trends in AI technology?"

[1303] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1304] Step 1:

[1305] The user inputs a keyword into the device. An example input is "latest AI technology trends." This input keyword becomes the basis for the subsequent data collection process. The input keyword is sent to the server.

[1306] Step 2:

[1307] Based on the keywords received from the user, the server calls social media APIs (e.g., Twitter API) and news site APIs (e.g., News API) to collect related data. Specifically, the server sends a query using the keywords to each API, and the retrieved data is returned to the server in JSON format. This data includes related posts and articles.

[1308] Step 3:

[1309] The server stores the collected data in a database (e.g., MongoDB or MySQL). A database is used to efficiently organize the data and store it in a searchable format. When storing, the JSON data is split into fields based on the appropriate schema and inserted.

[1310] Step 4:

[1311] The server performs pre-processing on the collected data. In this step, regular expression filters are used to remove advertisements and spam, and text normalization is performed. Specific operations include converting uppercase to lowercase, removing special characters, and deleting duplicates. The input is raw data retrieved from the database, and the output is the pre-processed, clean data.

[1312] Step 5:

[1313] The server analyzes the preprocessed data using natural language analysis techniques (e.g., morphological analysis, topic modeling, clustering). First, morphological analysis is performed to tokenize the text. Next, Latent Dirichlet Allocation (LDA) is used to identify major topics. Next, k-means clustering is used to group related topics. The input is the preprocessed data, and the output is a list of topics and clusters resulting from the analysis.

[1314] Step 6:

[1315] The server identifies frequently occurring keywords and phrases based on the results of natural language analysis and extracts major trends and buzzwords. In this step, important keywords are extracted and listed based on the results of topic modeling and clustering. The input is the analysis results, and the output is a list of extracted major trends and buzzwords.

[1316] Step 7:

[1317] The server uses an emotion analysis engine (e.g., VADemotion) to analyze the user's input and recognize their emotions. Specifically, when a user enters a phrase that expresses an emotion, such as "surprised," the server identifies that emotion and generates emotion data as the analysis result. The input is text related to the user's emotional mood, and the output is the recognized emotion data.

[1318] Step 8:

[1319] The server takes into account the results of the sentiment analysis and generates a summary for the user. Specifically, it formats the summary into something like, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." This summary is formatted in HTML. The input is the extracted trends, buzzwords, and sentiment data, and the output is the formatted summary.

[1320] Step 9:

[1321] The server sends the generated summary to the terminal. Specifically, the server sends summary data to the user's terminal using HTTP or the WebSocket protocol. The input is the summary in HTML format, and the output is the summary data received by the terminal.

[1322] Step 10:

[1323] The device parses the received summary and displays it to the user via a browser or dedicated application. Specifically, the device renders the received data in HTML format and displays it to the user. Through this interface, the user can check trends and buzzwords in real time. The user can also input new topics or keywords and request analysis again. The input is the received summary data, and the output is the information displayed to the user.

[1324] (Application example 2)

[1325] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1326] In today's information society, it is difficult for users to efficiently gather and understand the latest trend information. In particular, there is a need for a method to extract appropriate information from a wide range of information sources, such as social media and news sites, and provide it according to the user's interests. However, doing this manually takes a lot of time and effort. In addition, a system is needed that can provide more useful information by taking the user's emotions into consideration rather than simply collecting information.

[1327] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing collected data using natural language analysis technology, means for removing noise from the collected data, means for extracting related trends and buzzwords using topic modeling and clustering algorithms, means for performing sentiment analysis based on the analyzed trends and buzzwords, means for providing the user with a summary generated in consideration of the sentiment analysis, and means for displaying the generated summary on the user's terminal. This allows the user to efficiently obtain the latest trend information, and because the information is based on the user's emotions, more useful and comprehensive information can be provided.

[1328] "Natural language analysis technology" is a technology that uses computers to analyze text data and understand language structure.

[1329] "Noise" refers to unnecessary data or information that can cause inaccurate analysis results.

[1330] "Topic modeling" is a technique for extracting themes and themes from text data, and is a technology that analyzes data to identify potential topics.

[1331] A "clustering algorithm" is a mathematical technique for classifying data into groups, grouping related data points together.

[1332] "Sentiment analysis" is a technology that automatically extracts and analyzes user emotions and opinions from text data.

[1333] A "summary" summarizes the analyzed information and summarizes the main points.

[1334] "User terminal" refers to a communication device, such as a smartphone, tablet, or PC, that a user uses to obtain information.

[1335] This invention provides a system that efficiently collects real-time trend information based on keywords specified by a user, generates summaries taking into account sentiment analysis, and displays the summaries on the user's terminal. Specific embodiments of the system are described below.

[1336] System Configuration

[1337] This system is configured using the following hardware and software:

[1338] Hardware: Servers, user devices (smartphones, tablets, PCs)

[1339] Software: APIs (news site APIs, social media APIs), databases (MySQL, MongoDB), programming languages ​​(Python, JavaScript), NLP libraries (spaCy, NLTK), cloud services (AWS, Google Cloud)

[1340] Data collection

[1341] The server uses APIs of news sites and social media to collect related data based on keywords entered by the user on the device. This data includes text, images, videos, etc. For example, if a user enters "latest AI technology trends," the server retrieves the latest posts and articles related to "AI technology."

[1342] Data Preprocessing

[1343] The collected data is pre-processed to remove noise. Specifically, filtering algorithms are used to remove advertisements and spam, and text normalization is performed (for example, converting uppercase letters to lowercase and removing special characters). Duplicate data is also removed to improve the accuracy of the analysis.

[1344] Natural Language Analysis and Trend Extraction

[1345] After preprocessing, the data is tokenized using natural language analysis, topic modeling (e.g., Latent Dirichlet Allocation) is used to identify major topics, and then a clustering algorithm (e.g., k-means clustering) is used to group related topics and extract trends and buzzwords.

[1346] Emotion analysis

[1347] Based on the extracted trends and buzzwords, an emotion engine is used to analyze the user's emotions, thereby determining the emotions associated with the keywords entered by the user.

[1348] Summary generation and display

[1349] Taking into account the results of sentiment analysis, a summary optimized for the user is generated. For example, it may be formatted as, "The latest trends in AI technology are natural language processing and machine learning. Models such as GPT-4 and BERT are particularly popular." The generated summary is formatted in HTML and sent to the user's device. Users can check trend information that changes in real time and request reanalysis by entering new topics or keywords.

[1350] Specific examples

[1351] For example, if a user types "latest AI technology trends" into their device, the server will instantly collect relevant data from social media and news sites, perform sentiment analysis through the steps described above, and generate an optimal summary. Through this series of processes, users can efficiently obtain the latest trend information.

[1352] Prompt Sentence Examples

[1353] Collect data on the latest AI technology trends, perform sentiment analysis, and generate summaries based on user sentiment.

[1354] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1355] Step 1:

[1356] The user enters a trend keyword into the device. This entered keyword is sent to the server. For example, the user enters "latest AI technology trends." This keyword will become the basis for data collection in the next step.

[1357] Step 2:

[1358] The server collects related data using news site APIs and social media APIs based on the received keywords. Specifically, it retrieves posts and articles containing the user-entered "latest AI technology trends" via the API and stores the data in a database. The input is the keywords, and the output is the collected text data and metadata.

[1359] Step 3:

[1360] The server performs preprocessing to remove noise from the data stored in the database. Specifically, it uses filtering algorithms to remove advertisements and spam, and normalizes the text. At this stage, uppercase and lowercase letters are converted, special characters are removed, and duplicate data is deleted. The input is the collected text data, and the output is the preprocessed, clean text data.

[1361] Step 4:

[1362] The server tokenizes the preprocessed data using natural language processing (NLP) techniques, identifies major topics using topic modeling (e.g., Latent Dirichlet Allocation), and then groups related topics using a clustering algorithm (e.g., k-means clustering). The input is the preprocessed, clean text data, and the output is data for each topic and cluster.

[1363] Step 5:

[1364] Based on the topic and cluster information, the server analyzes the data using a sentiment analysis engine, which identifies user sentiment toward each topic or buzzword. Specifically, the server analyzes the text data and classifies it into sentiment categories such as positive, negative, and neutral. The input is data for each topic and cluster, and the output is the sentiment analysis results.

[1365] Step 6:

[1366] Based on the results of the sentiment analysis, the server generates a summary optimized for the user. For example, if a user is interested in "the latest AI technology trends," the server generates a summary such as, "The latest trends in AI technology are natural language processing and machine learning. In particular, models such as GPT-4 and BERT are popular." The input is the sentiment analysis results and topic information, and the output is a formatted summary.

[1367] Step 7:

[1368] The server sends the generated summary to the user's device, which then displays the summary. The user can check trend information changing in real time and can also input new keywords to request another analysis. The input is the generated summary, and the output is the summary displayed on the user's device.

[1369] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1370] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1371] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1372] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1373] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1374] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1375] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1376] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1377] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1378] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1379] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1380] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1381] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1382] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1383] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1384] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1385] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1386] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1387] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1388] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1389] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1390] The following is further disclosed regarding the above embodiment.

[1391] (Claim 1)

[1392] means for analyzing the collected data using natural language analysis techniques;

[1393] a means for removing noise from the collected data;

[1394] means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms;

[1395] means for collecting data based on user-specified keywords;

[1396] means for displaying the generated summary on a user's terminal;

[1397] A system including:

[1398] (Claim 2)

[1399] 2. The system of claim 1, wherein topic modeling uses Latent Dirichlet Allocation.

[1400] (Claim 3)

[1401] 2. The system according to claim 1, wherein the clustering algorithm uses the k-means algorithm.

[1402] "Example 1"

[1403] (Claim 1)

[1404] means for collecting data based on user-specified keywords;

[1405] a means for removing noise from the collected data;

[1406] means for pre-processing the collected data;

[1407] means for analyzing the preprocessed data using natural language analysis techniques;

[1408] means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms;

[1409] A means for generating summaries based on the extracted trends and buzzwords;

[1410] means for displaying the generated summary on a user's terminal;

[1411] A system including:

[1412] (Claim 2)

[1413] 2. The system of claim 1, wherein topic modeling uses Latent Dirichlet Allocation.

[1414] (Claim 3)

[1415] 2. The system according to claim 1, wherein the clustering algorithm uses the k-means algorithm.

[1416] "Application Example 1"

[1417] (Claim 1)

[1418] means for analyzing the collected data using natural language analysis techniques;

[1419] a means for removing noise from the collected data;

[1420] means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms;

[1421] means for collecting data based on user-specified keywords;

[1422] means for displaying the generated summary on a user's terminal;

[1423] A means of collecting user feedback to improve the accuracy of recommendations;

[1424] A system including:

[1425] (Claim 2)

[1426] 2. The system of claim 1, wherein topic modeling uses Latent Dirichlet Allocation.

[1427] (Claim 3)

[1428] 2. The system according to claim 1, wherein the clustering algorithm uses the k-means algorithm.

[1429] "Example 2: Combining Emotion Engines"

[1430] (Claim 1)

[1431] means for analyzing the collected data using natural language analysis techniques;

[1432] a means for removing noise from the collected data;

[1433] means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms;

[1434] means for collecting data from multiple sources based on user-specified keywords;

[1435] a means for integrating a sentiment analysis engine to understand user sentiment and adjust the generated summary accordingly;

[1436] means for transmitting the generated summary to a user's terminal and displaying it;

[1437] A system including:

[1438] (Claim 2)

[1439] 2. The system of claim 1, wherein topic modeling uses Latent Dirichlet Allocation.

[1440] (Claim 3)

[1441] 2. The system according to claim 1, wherein the clustering algorithm uses the k-means algorithm.

[1442] "Application example 2 when combining emotion engines"

[1443] (Claim 1)

[1444] means for analyzing the collected data using natural language analysis techniques;

[1445] a means for removing noise from the collected data;

[1446] means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms;

[1447] means for collecting data based on user-specified keywords;

[1448] A means for performing sentiment analysis based on the analyzed trends and buzzwords;

[1449] means for providing a user with a summary generated taking into account sentiment analysis;

[1450] means for displaying the generated summary on a user's terminal;

[1451] A system including:

[1452] (Claim 2)

[1453] 2. The system of claim 1, wherein topic modeling uses Latent Dirichlet Allocation.

[1454] (Claim 3)

[1455] 2. The system according to claim 1, wherein the clustering algorithm uses the k-means algorithm. [Explanation of symbols]

[1456] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for analyzing the collected data using natural language analysis techniques; a means for removing noise from the collected data; means for extracting relevant trends and buzzwords using topic modeling and clustering algorithms; means for collecting data based on user-specified keywords; means for displaying the generated summary on a user's terminal; A system including:

2. The system of claim 1 , wherein topic modeling uses Latent Dirichlet Allocation.

3. 2. The system according to claim 1, wherein the clustering algorithm uses a k-means algorithm.

Images