System

Abstract

A system is provided.SOLUTION: A system includes a data collection means, a data preprocessing means for cleansing collected data, a data analysis means for analyzing cleansed data, a knowledge extraction means for extracting important knowledge from an analysis result, and an information provision means for providing a user with the extracted knowledge.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, where there is a huge amount of data and diverse information sources, it is extremely important to quickly and accurately extract important insights from vast amounts of information and support user decision-making. However, traditional data collection and analysis methods are time-consuming and costly, and it is difficult to integrate and analyze information from diverse data sources. Furthermore, there are limited means to provide users with the insights they need in an appropriate format, making it difficult for them to effectively utilize information that is useful to them. These issues have hindered rapid and accurate decision-making. [Means for solving the problem]

[0005] The present invention provides a system including a data collection means, a data preprocessing means for cleansing collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, and an information provision means for providing the extracted knowledge to a user. In particular, the system includes a means for collecting information from specified data sources using an API, and analyzes the data using technologies such as natural language processing, image recognition, and voice recognition, thereby enabling rapid and accurate extraction of important knowledge from a variety of data sources. Furthermore, the extracted knowledge is provided in a customized format based on the user's settings and past trends, effectively supporting user decision-making. This reduces the time and cost required for information collection and analysis and enables the provision of useful insights to users.

[0006] A "data collection tool" is a tool for collecting information from designated data sources.

[0007] "Data pre-processing means" are means for cleansing collected data and preparing it for analysis.

[0008] "Data analysis means" refers to a means for analyzing cleansed data using AI algorithms.

[0009] "Knowledge extraction means" refers to a means for extracting important knowledge from the results of data analysis.

[0010] "Information provision means" refers to a means for providing extracted knowledge to users in an appropriate format.

[0011] "API" stands for Application Programming Interface, a set of rules for linking data and functions between different software.

[0012] "Natural language processing" is a technology that allows computers to understand, interpret, and generate human language.

[0013] "Image recognition" is the technology of identifying and locating objects and scenes within images.

[0014] "Speech recognition" is a technology that converts voice data into text and analyzes its content.

[0015] "User settings" refers to various customizations and parameter settings that a user makes to the system. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system including a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means. This system has a mechanism for collecting information from data sources specified by a user, analyzing the data, and providing useful insights.

[0038] Data collection methods:

[0039] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[0040] Data preprocessing methods:

[0041] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[0042] Data analysis methods:

[0043] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[0044] Knowledge extraction means:

[0045] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[0046] Information provision method:

[0047] Extracted insights are delivered to users in a customized format based on their preferences and historical trends, visualized on a dashboard, or sent to them as regular reports, allowing them to make quick and accurate decisions.

[0048] Specific examples

[0049] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server then uses the specified API to collect relevant information from these data sources. The collected data is then cleansed and sentiment analyzed using natural language processing. As a result, associations with times when there are many positive posts and specific keywords are discovered.

[0050] The server then uses these analyses to extract market trends and patterns in product reviews, such as graphs showing spikes in positive reviews or responses in specific market segments. This information is then sent to users in regular reports, allowing them to make decisions about marketing strategies and product improvements.

[0051] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and is able to quickly provide useful insights to users.

[0052] The processing flow will be explained below.

[0053] Step 1: User-specified data source configuration

[0054] Users tell the system the type of data they want to collect and the data source (e.g., social media platforms or news sites).

[0055] Step 2: Request to start data collection

[0056] The device sends a data collection request to the server based on the user's settings.

[0057] Step 3: Data Acquisition

[0058] The server calls the specified API (e.g., Twitter API, RSS feed) and collects the data. At this time, the URL and query parameters to be accessed are dynamically generated according to the user's settings.

[0059] Example: Collect related posts from the Twitter API based on the user-specified hashtag "market trends."

[0060] Step 4: Data cleansing

[0061] The server cleanses the collected data.

[0062] For text data: Remove HTML tags, unnecessary symbols, and links.

[0063] For image data: unify the resolution and reduce noise.

[0064] For audio data: Apply a noise reduction filter to improve audio clarity.

[0065] Step 5: Analytical processing according to the data format

[0066] The server passes the cleansed data to various AI algorithms for analysis.

[0067] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction from text data.

[0068] Image recognition algorithms are used to recognize specific objects and scenes.

[0069] Using a speech recognition algorithm, the voice data is converted into text and further subjected to sentiment analysis.

[0070] Step 6: Find data correlations

[0071] The server analyzes the relationships between different data sources and extracts important correlations.

[0072] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[0073] Step 7: Creating a visualization

[0074] The server generates dashboards and graphs to visually illustrate the extracted correlations.

[0075] Example: Graphing sentiment analysis results on a timeline.

[0076] Step 8: Generate reports for users

[0077] The server compiles insights from user settings and historical data to create customized reports.

[0078] Example: Generate a weekly report summarizing market trends and sentiment analysis related to a specified hashtag.

[0079] Step 9: Notification and distribution

[0080] The device will notify the user of important announcements and analysis results at specific times.

[0081] Example: Sending reports generated by the server to users via email.

[0082] Example: Analytical results are updated in real time on a dashboard, providing users with a view into the results.

[0083] Example 1

[0084] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0085] Conventional data analysis systems have struggled to provide users with the information they need quickly and accurately, particularly in the complex and time-consuming process of consistently preprocessing data collected from different data sources and extracting useful insights in real time to provide them to users.

[0086] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0087] In this invention, the server includes a data collection means for collecting information from specified data sources using APIs, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data using natural language processing, image recognition, and voice recognition, a knowledge extraction means for identifying correlations between different data sources and specific patterns from the analysis results and extracting important knowledge, and an information provision means for providing the extracted knowledge in a customized format based on user settings and past trends, visualizing it on a dashboard, or sending it to the user as a regular report, thereby enabling the user to quickly and accurately obtain useful insights.

[0088] Below are definitions of key words.

[0089] The "terminal means" is a device for inputting a data source and a request designated by a user and transmitting the input to a server.

[0090] "Data collection means" means a means of obtaining necessary information from a specified data source using an API.

[0091] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary symbols and links, and arranging the data into an appropriate format.

[0092] "Data analysis means" refers to means for analyzing cleansed data using natural language processing, image recognition, and voice recognition.

[0093] "Insight extraction means" refers to means for identifying correlations and specific patterns between different data sources from the results of data analysis and extracting important insights.

[0094] "Information provision means" refers to the means of providing extracted knowledge to users, visualizing it on a dashboard, or sending it as a regular report.

[0095] "API" is an abbreviation for Application Programming Interface, an interface for exchanging data between different software programs.

[0096] "Natural language processing" is a technology that uses computers to understand and process human language.

[0097] "Image recognition" is a technology that identifies specific objects from digital images and videos.

[0098] "Speech recognition" is a technology that converts voice data into text and understands its content.

[0099] A "dashboard" is an interface that provides users with visualized data information.

[0100] A "report" is an information document that summarizes analysis results and findings in written form and provides them to users.

[0101] The present invention provides a system for collecting information from data sources specified by a user, analyzing the data, and providing useful insights. The system includes a terminal unit, a data collection unit, a data preprocessing unit, a data analysis unit, a knowledge extraction unit, and an information provision unit.

[0102] System configuration and operation

[0103] Terminal means

[0104] The user uses the device interface to specify a particular data source (e.g., a social media platform, a news site) and associated hashtags, and this information is sent to the server in the form of a request.

[0105] Data collection methods

[0106] The server uses APIs to gather information from data sources specified by the user, for example, using the Twitter API or a news site's API to gather posts and articles related to a particular hashtag.

[0107] Data preprocessing measures

[0108] The server cleanses the collected data. Specifically, it removes unnecessary symbols and links from text data, standardizes the resolution of image data, and removes noise. It also performs noise reduction and segmentation on audio data. For example, it cleanses text data using the Python pandas library.

[0109] Data Analysis Methods

[0110] The cleansed data is then analyzed by the server. Natural language processing (NLP) is used to perform sentiment analysis on the text data, and image recognition is used to identify specific objects or scenes within images. For audio data, a speech recognition algorithm is used to convert it into text and analyze its content. Specifically, Google's BERT model is used for natural language processing, and OpenCV is used for image recognition.

[0111] Knowledge extraction means

[0112] From the analysis results, the server identifies correlations and specific patterns between different data sources and extracts key insights, such as the correlation between sentiment scores of social media posts and stock prices in market data. This process is powered by the Scikit-learn library.

[0113] Information provision means

[0114] The extracted insights are delivered in a customized format based on user preferences and historical trends, visualized on a dashboard, or sent to users as periodic reports. For example, the dashboards are generated using Tableau, and the reports are generated using Python's ReportLab.

[0115] Specific examples

[0116] If a user wants to analyze the market valuation of a new product, they would follow these steps:

[0117] 1. Users enter the name of a new product and related hashtags into the system and specify Twitter or a news site as the data source.

[0118] 2. The server collects relevant information from these data sources using the specified APIs.

[0119] 3. After the collected data is cleansed, sentiment analysis is performed using natural language processing. For example, Twitter posts are analyzed using the BERT model to classify them as positive, negative, or neutral.

[0120] 4. The server uses these analysis results to extract market trends and patterns of product reviews. It then creates graphs to visualize periods of time when positive reviews spike and reactions in specific market segments. Matplotlib is used to generate these graphs.

[0121] 5. Extracted insights are sent to users in regular reports, with specific insights such as, "Positive posts about new products are most prevalent between 2:00 PM and 4:00 PM."

[0122] Prompt Sentence Examples

[0123] I'd like to analyze the market evaluation of new product XYZ, so I'd like you to collect posts related to XYZ from Twitter and news sites and perform a sentiment analysis. For example, I'd like to know how many positive and negative posts there are.

[0124] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, making it possible to quickly provide useful insights to users.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1:

[0127] User request submission

[0128] The user uses the device interface to specify a specific data source (e.g., a social media platform, a news site) and relevant hashtags. Once the user enters this information and presses the send button, the request is sent from the device to the server.

[0129] Input: Data source and hashtags entered by the user into the form (e.g., "Twitter" or "New product name")

[0130] Output: Request data from the terminal to the server

[0131] Specific behavior:

[0132] A user enters the name of a new product and related hashtags into a form.

[0133] The user selects Twitter or a news site.

[0134] The user presses the "Send" button.

[0135] The terminal transmits the request data to the server.

[0136] Step 2:

[0137] Data collection by the server

[0138] Based on the request received, the server uses APIs to gather information from specified data sources, for example, Twitter API or a news site's API to retrieve posts and articles related to a specific hashtag.

[0139] Input: Request data sent from the terminal

[0140] Output: Raw data obtained from each data source

[0141] Specific behavior:

[0142] The server accesses the Twitter API and collects posts containing "new product name."

[0143] The server accesses the news site's API and collects articles related to the keyword "new product name."

[0144] The server stores this raw data internally.

[0145] Step 3:

[0146] Data preprocessing by the server

[0147] The server cleanses the collected data by removing unnecessary symbols and links from text data, standardizing the resolution of image data and removing noise, and performing noise reduction and segmentation on audio data.

[0148] Input: Raw data obtained from each data source

[0149] Output: Cleansed data

[0150] Specific behavior:

[0151] The server removes unnecessary links and symbols from the Twitter posts it retrieves.

[0152] The server extracts the body of the news article and removes any advertisements or sidebar information.

[0153] The server standardizes the resolution of the image data and removes noise.

[0154] Step 4:

[0155] Data analysis by server

[0156] The server analyzes the pre-processed data: natural language processing (NLP) is used to perform sentiment analysis on text data, image recognition is used to identify specific objects on image data, and speech recognition algorithms are used to convert audio data into text and analyze its content.

[0157] Input: Cleansed data

[0158] Output: Analysis result data

[0159] Specific behavior:

[0160] The server performs sentiment analysis on the Twitter posts retrieved by the server using the BERT model, classifying them as positive, negative, or neutral.

[0161] Analyze the titles and text of news articles to extract trending topics and keywords.

[0162] The server performs object recognition on the image data using OpenCV.

[0163] The audio data is converted into text and the content is further analyzed.

[0164] Step 5:

[0165] Knowledge extraction by server

[0166] The server extracts key findings from the data analysis, finding correlations and specific patterns between different data sources and generating useful insights.

[0167] Input: Analysis result data

[0168] Output: Knowledge data

[0169] Specific behavior:

[0170] The server compares the Twitter sentiment analysis results with market stock price data to find correlations between the two.

[0171] The increase or decrease in positive posts during specific time periods is graphed, suggesting applications for marketing.

[0172] Step 6:

[0173] Information provided by the server

[0174] The server provides the extracted insights to the user, visualizing them on a dashboard or sending them to the user as regular reports. Specifically, it generates graphs and tables to visualize the insights and provides the insights in a format that is easy for the user to understand.

[0175] Input: Knowledge data

[0176] Output: Customization information provided to the user

[0177] Specific behavior:

[0178] The server generates a graph of times when there are many positive posts and displays it on the dashboard.

[0179] Generate a weekly report in PDF format and send it to your email address.

[0180] In this way, the system can perform clear steps and provide useful insights to the user quickly.

[0181] (Application example 1)

[0182] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0183] Conventional surveillance systems have difficulty in monitoring a wide area or detecting suspicious individuals or objects in real time. Furthermore, there have been no systems that provide risk information by integrating information collection from social media and sentiment analysis. This has made it difficult to make quick and accurate decisions, and has led to problems in implementing effective security measures.

[0184] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0185] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, a video recognition means for monitoring video data in real time and detecting suspicious objects and people, and a natural language processing means for collecting risk information from social media and performing sentiment analysis. This enables wide-area monitoring and real-time detection of suspicious people and objects, and also enables risk information to be provided by integrating social media information, enabling quick and accurate decision-making.

[0186] A "data collection method" is a method by which a user collects information from a particular data source.

[0187] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary information, and preparing the data in an appropriate format.

[0188] "Data analysis means" refers to means for analyzing cleansed data to extract specific patterns and trends.

[0189] "Knowledge extraction means" refers to a means for extracting important knowledge from the results of data analysis.

[0190] "Information provision means" refers to a means for providing extracted knowledge to users.

[0191] "Video recognition means" is a means for monitoring video data in real time and detecting suspicious objects and people.

[0192] "Natural language processing means" is a means for collecting risk information from social media and performing sentiment analysis.

[0193] The present invention aims to realize a security service that collects information from data sources specified by a user, analyzes the information, and provides useful insights. The system includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means.

[0194] Data collection methods:

[0195] The server collects real-time video from devices such as surveillance cameras and drones. This video data is used to detect suspicious objects and people. Additionally, posts related to designated hashtags on social media (e.g., Twitter) can be collected via API. This allows users to efficiently collect a wide range of information.

[0196] Data preprocessing methods:

[0197] The server cleanses the collected video and social media data. For video data, it standardizes image quality and removes noise, and for text data, it removes unnecessary symbols and links. This improves the accuracy of the data and increases the efficiency of subsequent analysis.

[0198] Data analysis methods:

[0199] The server applies video recognition algorithms and natural language processing (NLP) to the cleansed data. Video recognition is used to detect suspicious objects and people in real time from surveillance camera footage. NLP is also used to perform sentiment analysis of social media posts and extract potential risk information. These analyses are performed using existing libraries such as OpenCV and TextBlob.

[0200] Knowledge extraction means:

[0201] The server extracts important insights from the analysis results. For example, it integrates location information of suspicious objects detected from video data and risk information extracted from social media, and finds correlations. This allows users to make accurate decisions in real time.

[0202] Information provision method:

[0203] The server uses a head-mounted display (HMD) to provide extracted insights to users, showing the location of suspicious objects and people in real time and providing social media sentiment analysis results as warnings, allowing users to take prompt and appropriate action.

[0204] Examples:

[0205] For example, if a security camera monitors an area in real time and detects a suspicious individual, this information will be sent to security guards via the HMD. Meanwhile, if social media posts related to the same area are analyzed and any disturbing information is found, this information will also be sent to security guards. This allows security guards to grasp the overall situation and respond quickly.

[0206] Example prompt sentence:

[0207] Below is an example of a prompt for a system using the present invention:

[0208] "I want to create an AI model that collects video data from surveillance cameras in real time and detects suspicious people and objects. I want to preprocess the collected video data and analyze it using the AI ​​model. How can I then build a system that displays the location information and warnings of detected suspicious objects in real time?"

[0209] "I want to collect posts related to a specified hashtag on Twitter and perform sentiment analysis. How can I build a system that extracts risk information in a specific area and issues real-time alerts to security guards?"

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] Data collection

[0213] The server acquires real-time video data from devices such as surveillance cameras and drones. The input is a video data source such as a surveillance camera, and the server captures and saves it frame by frame. In parallel, it uses the specified social media API to collect posts containing related hashtags and keywords. In this case, the input is the hashtag or keyword, and the output is the related SNS post data.

[0214] Step 2:

[0215] Data Preprocessing

[0216] The server cleanses the collected video data. Specific operations include unifying the resolution of the video data, removing noise, and improving image quality. The input is the video data collected in step 1, and the output is the preprocessed video data. For social media data, unnecessary symbols and links are deleted and text data is cleaned. The input is text data collected from SNS, and the output is preprocessed text data.

[0217] Step 3:

[0218] Data analysis

[0219] The server applies a video recognition algorithm to the preprocessed video data. Specifically, it performs object detection to detect suspicious objects and people for each frame. The input is the preprocessed video data, and the output is information about the detected suspicious objects and people. In addition, it performs sentiment analysis on the preprocessed text data using natural language processing (NLP). This allows it to obtain positive or negative sentiment scores as output from the processed text data as input.

[0220] Step 4:

[0221] Knowledge extraction

[0222] The server extracts important insights from the results of the data analysis. Specifically, it extracts location information of suspicious objects detected from video data and behavioral patterns of suspicious individuals. The input is the analysis results obtained in step 3, and the output is the extracted insights (detailed information on suspicious objects and individuals). Similarly, it extracts posts with high negative scores in specific regions from the results of social media sentiment analysis.

[0223] Step 5:

[0224] Providing information

[0225] The server uses a head-mounted display (HMD) to provide the extracted insights to the user. Specifically, it displays the location information of suspicious objects and people on the HMD in real time and issues warnings based on the results of social media sentiment analysis. The input is the insights extracted in step 4, and the output is the information and warnings displayed on the HMD. This allows the user to take prompt and appropriate action.

[0226] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0227] The present invention is a system that includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means, as well as an emotion engine that recognizes user emotions. This system has a mechanism for collecting information from data sources specified by the user, analyzing that data, and providing useful insights. It also has a function for customizing the information provided by taking the user's emotions into consideration.

[0228] Data collection methods:

[0229] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[0230] Data preprocessing methods:

[0231] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[0232] Data analysis methods:

[0233] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[0234] Knowledge extraction means:

[0235] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[0236] Information provision method:

[0237] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the information is customized based on the user's sentiment, and visualized on a dashboard or sent to the user as regular reports. Users can then make quick and accurate decisions based on this information.

[0238] Emotion Engine:

[0239] It has the ability to recognize the user's emotions in real time and adjust the information it provides based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that will reduce stress.

[0240] Specific examples

[0241] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[0242] For example, if the emotion engine detects that a user is feeling anxious, it will create a report centered around positive reviews and success stories, providing information that will reduce the user's stress, allowing the user to make decisions about marketing strategies and product improvements with peace of mind.

[0243] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and further customizes information taking into account the user's emotions, making it possible to quickly provide the most useful insights to the user.

[0244] The processing flow will be explained below.

[0245] Step 1: Specify the data source

[0246] Users tell the system the type of data they want to collect (e.g., text, images, audio) and the corresponding data source (e.g., social media platform, news site).

[0247] Step 2: Submit a request for data collection

[0248] The device sends a data collection request to the server based on the user's specifications, including the data source URL and query parameters.

[0249] Step 3: Collect data

[0250] The server calls the specified API (e.g., Twitter API, RSS feed) to collect data, which is then temporarily stored.

[0251] Step 4: Cleanse the data

[0252] The server cleanses the collected data.

[0253] For text data: Remove HTML tags, unnecessary symbols, and links and purify the data into text.

[0254] For image data: The resolution is unified and noise is removed.

[0255] For audio data: Apply a noise reduction filter to improve audio clarity.

[0256] Step 5: Data analysis

[0257] The server passes the cleansed data to various AI algorithms for analysis.

[0258] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction of text data.

[0259] Image recognition algorithms identify specific objects and scenes.

[0260] A speech recognition algorithm converts the voice data into text and then analyzes the content.

[0261] Step 6: Extracting insights

[0262] The server extracts key insights from the results of data analysis, which includes finding correlations between different data sources and pattern recognition.

[0263] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[0264] Step 7: Recognizing user emotions

[0265] The emotion engine analyzes the user's real-time emotional state, including emotion analysis from voice data and emotion inference from user input behavior.

[0266] Step 8: Customize your information offering

[0267] The server adjusts the information provided to the user based on the emotional state obtained by the emotion engine.

[0268] Example: If a user is in an anxious state, prioritize positive information.

[0269] Step 9: Provide information

[0270] The server provides tailored insights to users, including visualization in a dashboard and the generation and delivery of periodic reports.

[0271] Information is updated in real time on the dashboard and displayed visually in an easy-to-understand manner.

[0272] The analysis results will be emailed to the user as a weekly report.

[0273] Step 10: User feedback

[0274] Users make decisions based on the information provided and input their results and feedback into the system, which allows the system to learn and reflect this in the next information provided.

[0275] The above is the specific processing flow in the system of the present invention. In this way, it becomes possible to provide useful insights to users quickly and accurately.

[0276] Example 2

[0277] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0278] Conventional data analysis systems process and provide collected data in a uniform manner, which means they are unable to consider the emotional state or individual needs of users and are unable to support optimal decision-making.In addition, the wide variety of data sources makes it difficult to efficiently cleanse and analyze collected data.

[0279] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0280] In this invention, the server includes a data collection means, a preprocessing means for cleansing the collected data, an analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing the user's emotions and adjusting information based on that state. This makes it possible to provide information customized according to the user's emotional state, thereby supporting faster and more effective decision-making.

[0281] "Data Collection Implement" means any device or software used to collect information from user-specified data sources.

[0282] A "pre-processing means" is a device or software that cleanses the collected data and prepares it in a form suitable for analysis.

[0283] "Analysis means" refers to a device or software that analyzes the cleansed data using various algorithms to extract useful information and patterns.

[0284] The "knowledge extraction means" is a device or software for extracting important insights and meaningful knowledge from the analysis results obtained by the analysis means.

[0285] "Information provision means" refers to devices or software that provide extracted knowledge to users in an easy-to-understand manner. Specifically, this includes visualization on a dashboard and generation of reports.

[0286] An "emotion engine" is a device or software that recognizes a user's emotional state in real time and adjusts the information provided based on that state.

[0287] "API" stands for Application Programming Interface, a standardized way of sending and receiving data between different software applications.

[0288] "Natural language processing" is the technology for understanding and analyzing human language, and is used for sentiment analysis and semantic analysis of text.

[0289] "Image recognition" is the technology of identifying specific objects or patterns from image data.

[0290] "Speech recognition" is a technology that converts voice data into text and analyzes its content.

[0291] The present invention is a system that includes a data collection means, a preprocessing means, an analysis means, a knowledge extraction means, an information provision means, and an emotion engine. This allows the system to collect information from a data source specified by a user, analyze the data, and provide useful knowledge. The system also has a function to adjust the information taking the user's emotions into account.

[0292] Data collection methods

[0293] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies a specific hashtag, the server retrieves posts related to that hashtag. This collected data is then stored in the server's internal database.

[0294] Pretreatment means

[0295] The collected data is cleansed by the server. In the case of text data, unnecessary symbols and links are removed and the data is formatted appropriately. In the case of image data, the resolution is standardized and noise is removed. In the case of audio data, noise reduction and segmentation are performed. This preprocessing converts the data into a format suitable for analysis.

[0296] analytical means

[0297] The cleansed data is then analyzed by the server using various AI algorithms. Specifically, natural language processing (NLP) is used to perform sentiment analysis on text data, and image recognition is used to identify specific objects or scenes in images. Voice data is converted into text using speech recognition algorithms, and its content is analyzed. These algorithms are implemented using generative AI models.

[0298] Knowledge extraction means

[0299] The server extracts key insights from the analysis results, for example, by finding correlations between different data sources and analyzing the correlation between sentiment scores of social media posts and stock prices from market data, allowing users to gain deeper insights.

[0300] Information provision means

[0301] The extracted insights are delivered in a customized format based on the user's settings and past trends. The server visualizes these insights on a dashboard or sends them to the user as regular reports, allowing the user to make quick and accurate decisions based on the information provided.

[0302] Emotion Engine

[0303] The emotion engine recognizes the user's emotions in real time and adjusts the information provided based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress. This allows the system to provide the most appropriate information for the user.

[0304] Specific examples

[0305] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies social media platforms and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[0306] Prompt Sentence Examples

[0307] For example, a user might prompt a generative AI model as follows:

[0308] "I want to analyze the market evaluation of a new product using social media platforms and news sites as data sources. The related hashtag is new product 2023. Based on the results of the sentiment engine, please create a report with positive evaluations and success stories."

[0309] In this way, by automating the entire process from data collection to providing insights, and further customizing information taking into account user emotions, we have created a system that can quickly provide the most useful insights to users.

[0310] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0311] Step 1:

[0312] The user specifies a particular data source.

[0313] Users enter the required keywords or hashtags into the system's interface.

[0314] For example, a user enters the hashtag "New Products 2023."

[0315] Input data: hashtag "New Products 2023"

[0316] Output: API request data as a user request

[0317] Specific operation: When the user clicks the search button, the device converts the entered data into an API request format and sends it to the server.

[0318] Step 2:

[0319] The terminal sends the user's request to the server.

[0320] The device converts the request, including the hashtags and keywords entered by the user, into JSON format.

[0321] Input data: User-entered keywords (API request data)

[0322] Output: API request sent to the server

[0323] Specific operation: The device sends an API request to the server in the form of an HTTP request.

[0324] Step 3:

[0325] The server uses the specified API to collect information from the data source.

[0326] The server sends a request to the API endpoint of the social media platform or news site to retrieve the required data.

[0327] Input data: API request

[0328] Output: Data collected from social media platforms and news sites (text, images, audio, etc.)

[0329] Specific operation: The server sends a request to the API endpoint and stores the returned JSON data in an internal database.

[0330] Step 4:

[0331] The server cleanses the collected data.

[0332] The server removes unnecessary symbols and links from the text data and formats it appropriately. For image data, it standardizes the resolution and removes noise. For audio data, it performs noise reduction and segmentation.

[0333] Input data: raw data collected

[0334] Output: Cleansed data (clean text, images, audio data)

[0335] Specific operation: The server executes the data cleansing algorithm, filtering and format conversion.

[0336] Step 5:

[0337] The server analyzes the cleansed data.

[0338] The server uses natural language processing (NLP) to perform sentiment analysis on the text data, image recognition to identify specific objects and scenes in images, and speech recognition algorithms to convert audio data into text and analyze it.

[0339] Input data: Cleansed data (text, image, audio data)

[0340] Output: Analysis results (emotion scores, recognition results, text conversion data, etc.)

[0341] Specific operation: The server runs NLP algorithms and image recognition algorithms to generate emotion scores and object identification results.

[0342] Step 6:

[0343] The server extracts key insights from the analysis results.

[0344] The server analyzes correlations between different data sources and extracts key insights and findings.

[0345] Input data: Analysis results

[0346] Output: Extracted findings (correlations, patterns, insights, etc.)

[0347] What it does: The server analyzes the analysis results in the database and runs algorithms to extract key insights.

[0348] Step 7:

[0349] The server customizes the insights to provide to the user.

[0350] The server customizes the extracted insights based on the user's past preferences and tendencies.

[0351] Input data: extracted knowledge, user setting data

[0352] Output: Customized insights (reports and dashboards tailored to the user)

[0353] Specific operation: The server retrieves the user's profile data and uses the extracted insights to customize it.

[0354] Step 8:

[0355] The server uses an emotion engine to recognize the user's emotions in real time.

[0356] The emotion engine analyzes the user's emotional state from their facial expressions and voice.

[0357] Input data: Real-time facial expression data and voice data of the user

[0358] Output: User's emotional state (positive, negative, etc.)

[0359] Specific operation: The emotion engine runs facial expression recognition algorithms and voice analysis algorithms to evaluate the user's emotions.

[0360] Step 9:

[0361] The server tailors the information based on the user's emotional state.

[0362] The server adjusts the content of the information it provides depending on the recognized emotional state.

[0363] Input data: user's emotional state, customized insights

[0364] Output: Emotionally adjusted information (reports and feedback containing positive information)

[0365] Specific operation: The server filters information based on the emotional state, selects appropriate information and provides it to the user.

[0366] Step 10:

[0367] The server provides the information to the user.

[0368] The server visualizes the information on a dashboard or sends it to the user as periodic reports.

[0369] Input data: emotion-modulated information

[0370] Output: The final information provided to the user (dashboard display, report sending, etc.)

[0371] What it does: The server generates graphs and tables to visualize the information and displays them on the user's dashboard. It also generates a report in PDF format and emails it to the user.

[0372] This will automate the entire process from data collection to providing insights, and by customizing information taking into account user emotions, a system will be created that supports quick and accurate decision-making.

[0373] (Application example 2)

[0374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional data collection and analysis systems are unable to provide information that takes into account the user's emotions, which can result in users receiving inappropriate information. Furthermore, there is no system in place to recognize customer emotions in real time and respond appropriately based on those emotions, making it difficult for store staff to respond quickly and accurately to customer needs.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0377] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing customer emotions in real time and customizing the information to be provided. This makes it possible to provide the most appropriate information to the user and to present an appropriate response method according to the customer's emotions.

[0378] 1. "Data Collection Implement" means a device or system capable of collecting information from user-specified data sources.

[0379] 2. "Data preprocessing means" refers to a device or system that removes unnecessary information and cleanses data in order to prepare collected data in an appropriate format.

[0380] 3. "Data Analysis Tool" means a device or system that uses AI algorithms to analyze cleansed data and extract useful insights.

[0381] 4. A "Insight Extraction Tool" is a device or system used to find significant patterns and correlations from the results of data analysis.

[0382] 5. "Information Delivery Measure" means a device or system that delivers extracted insights to a user in a customized format based on the user's preferences and past trends.

[0383] 6. An "emotion engine" is a device or system that has the ability to recognize the emotions of users or customers in real time and adjust the information it provides based on that state.

[0384] 7. "API" means an interface for exchanging functions and data between different software programs, and is used to collect information from specified data sources.

[0385] 8. "Natural language processing" is a technology for analyzing text data and performing sentiment analysis and understanding intent.

[0386] 9. “Image recognition” is a technology for identifying specific objects or scenes from image data.

[0387] 10. "Speech recognition" is a technology for converting voice data into text and analyzing its content.

[0388] The system of the present invention mainly includes the following means: data collection means, data preprocessing means, data analysis means, knowledge extraction means, information provision means, and an emotion engine. This system has a mechanism for collecting information from data sources specified by the user, analyzing the data, and providing useful insights. It also has a function for recognizing the emotions of users and customers in real time and customizing the information provided based on those emotions.

[0389] First, when a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the necessary information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[0390] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[0391] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using speech recognition algorithms, and its content is then further analyzed.

[0392] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[0393] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the emotion engine recognizes the user's and customer's emotions in real time and adjusts the information provided based on their state. For example, if a user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress.

[0394] A specific example is when store staff use their smartphones to capture customers' facial expressions and tone of voice while serving them, and then suggest appropriate ways to respond based on the analysis results. This allows staff to suggest products and provide services that are appropriate to the customer's emotions.

[0395] Example prompt sentence:

[0396] "Identify the specific emotion this customer is experiencing and, based on that, suggest the best product or customer service approach."

[0397] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0398] Step 1:

[0399] The user specifies a specific data source (e.g., a social media platform or a news site) and sends a request to the device.

[0400] Input: User-specified data source and associated parameters (e.g., hashtags)

[0401] Output: Request data from the terminal to the server

[0402] How it works: The user interacts with the application and inputs the data source to be investigated and related parameters. The device then sends this information to the server.

[0403] Step 2:

[0404] The server uses the API to collect information from the specified data sources.

[0405] Input: Request data sent from the terminal

[0406] Output: Raw data retrieved from the specified data source.

[0407] What it does: The server makes an API request to gather information from the specified data source (e.g., Twitter API), including related posts and articles.

[0408] Step 3:

[0409] The server cleanses the collected data.

[0410] Input: Raw data collected

[0411] Output: Cleansed data

[0412] How it works: The server removes unnecessary symbols and links from text data, standardizes the resolution of image data, and reduces noise in audio data, thereby standardizing the data.

[0413] Step 4:

[0414] The server analyzes the cleansed data.

[0415] Input: Cleansed data

[0416] Output: Analysis results (e.g., sentiment scores, identified objects)

[0417] How it works: The server uses natural language processing (NLP) algorithms to perform sentiment analysis on text data, image data to analyze using image recognition algorithms, and audio data to convert it into text using speech recognition algorithms, which then analyzes its content.

[0418] Step 5:

[0419] The server extracts important insights from the analysis results.

[0420] Input: Analysis results

[0421] Output: Key findings (e.g. correlations, specific patterns)

[0422] How it works: The server finds correlations between different data sources and identifies specific patterns, such as the correlation between sentiment scores from social media posts and stock prices from market data.

[0423] Step 6:

[0424] The server provides the extracted knowledge to the user.

[0425] Input: Key Findings

[0426] Output: Customized information (e.g. reports, dashboards)

[0427] How it works: The server customizes insights based on user preferences and historical trends, visualizes them on a dashboard, or sends them to the user as regular reports.

[0428] Step 7:

[0429] The server uses an emotion engine to recognize the user's emotions in real time and adjusts information based on that state.

[0430] Input: User emotion data (e.g., facial expressions, tone of voice)

[0431] Output: Tailored information

[0432] How it works: The server uses a camera and microphone to capture and analyze the customer's facial expressions and tone of voice. Based on the analysis results, it provides information that corresponds to the user's emotional state. For example, if the user is anxious, it prioritizes positive information.

[0433] Step 8:

[0434] Users and store staff receive the information provided by the server and apply it to their actual work.

[0435] Input: Customized information provided by the server

[0436] Output: Real-world decisions and actions

[0437] How it works: Users and store staff decide business policies based on reports and dashboards provided by the server, and reflect these in customer service and marketing strategies.

[0438] This allows users to receive relevant insights in a timely manner, enabling them to provide better services and make better decisions based on customer sentiment.

[0439] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0440] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0441] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0442] [Second embodiment]

[0443] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0444] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0445] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0446] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0447] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0448] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0449] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0450] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0451] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0452] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0453] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0454] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0455] The present invention is a system including a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means. This system has a mechanism for collecting information from data sources specified by a user, analyzing the data, and providing useful insights.

[0456] Data collection methods:

[0457] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[0458] Data preprocessing methods:

[0459] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[0460] Data analysis methods:

[0461] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[0462] Knowledge extraction means:

[0463] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[0464] Information provision method:

[0465] Extracted insights are delivered to users in a customized format based on their preferences and historical trends, visualized on a dashboard, or sent to them as regular reports, allowing them to make quick and accurate decisions.

[0466] Specific examples

[0467] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server then uses the specified API to collect relevant information from these data sources. The collected data is then cleansed and sentiment analyzed using natural language processing. As a result, associations with times when there are many positive posts and specific keywords are discovered.

[0468] The server then uses these analyses to extract market trends and patterns in product reviews, such as graphs showing spikes in positive reviews or responses in specific market segments. This information is then sent to users in regular reports, allowing them to make decisions about marketing strategies and product improvements.

[0469] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and is able to quickly provide useful insights to users.

[0470] The processing flow will be explained below.

[0471] Step 1: User-specified data source configuration

[0472] Users tell the system the type of data they want to collect and the data source (e.g., social media platforms or news sites).

[0473] Step 2: Request to start data collection

[0474] The device sends a data collection request to the server based on the user's settings.

[0475] Step 3: Data Acquisition

[0476] The server calls the specified API (e.g., Twitter API, RSS feed) and collects the data. At this time, the URL and query parameters to be accessed are dynamically generated according to the user's settings.

[0477] Example: Collect related posts from the Twitter API based on the user-specified hashtag "market trends."

[0478] Step 4: Data cleansing

[0479] The server cleanses the collected data.

[0480] For text data: Remove HTML tags, unnecessary symbols, and links.

[0481] For image data: unify the resolution and reduce noise.

[0482] For audio data: Apply a noise reduction filter to improve audio clarity.

[0483] Step 5: Analytical processing according to the data format

[0484] The server passes the cleansed data to various AI algorithms for analysis.

[0485] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction from text data.

[0486] Image recognition algorithms are used to recognize specific objects and scenes.

[0487] Using a speech recognition algorithm, the voice data is converted into text and further subjected to sentiment analysis.

[0488] Step 6: Find data correlations

[0489] The server analyzes the relationships between different data sources and extracts important correlations.

[0490] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[0491] Step 7: Creating a visualization

[0492] The server generates dashboards and graphs to visually illustrate the extracted correlations.

[0493] Example: Graphing sentiment analysis results on a timeline.

[0494] Step 8: Generate reports for users

[0495] The server compiles insights from user settings and historical data to create customized reports.

[0496] Example: Generate a weekly report summarizing market trends and sentiment analysis related to a specified hashtag.

[0497] Step 9: Notification and distribution

[0498] The device will notify the user of important announcements and analysis results at specific times.

[0499] Example: Sending reports generated by the server to users via email.

[0500] Example: Analytical results are updated in real time on a dashboard, providing users with a view into the results.

[0501] Example 1

[0502] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0503] Conventional data analysis systems have struggled to provide users with the information they need quickly and accurately, particularly in the complex and time-consuming process of consistently preprocessing data collected from different data sources and extracting useful insights in real time to provide them to users.

[0504] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0505] In this invention, the server includes a data collection means for collecting information from specified data sources using APIs, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data using natural language processing, image recognition, and voice recognition, a knowledge extraction means for identifying correlations between different data sources and specific patterns from the analysis results and extracting important knowledge, and an information provision means for providing the extracted knowledge in a customized format based on user settings and past trends, visualizing it on a dashboard, or sending it to the user as a regular report, thereby enabling the user to quickly and accurately obtain useful insights.

[0506] Below are definitions of key words.

[0507] The "terminal means" is a device for inputting a data source and a request designated by a user and transmitting the input to a server.

[0508] "Data collection means" means a means of obtaining necessary information from a specified data source using an API.

[0509] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary symbols and links, and arranging the data into an appropriate format.

[0510] "Data analysis means" refers to means for analyzing cleansed data using natural language processing, image recognition, and voice recognition.

[0511] "Insight extraction means" refers to means for identifying correlations and specific patterns between different data sources from the results of data analysis and extracting important insights.

[0512] "Information provision means" refers to the means of providing extracted knowledge to users, visualizing it on a dashboard, or sending it as a regular report.

[0513] "API" is an abbreviation for Application Programming Interface, an interface for exchanging data between different software programs.

[0514] "Natural language processing" is a technology that uses computers to understand and process human language.

[0515] "Image recognition" is a technology that identifies specific objects from digital images and videos.

[0516] "Speech recognition" is a technology that converts voice data into text and understands its content.

[0517] A "dashboard" is an interface that provides users with visualized data information.

[0518] A "report" is an information document that summarizes analysis results and findings in written form and provides them to users.

[0519] The present invention provides a system for collecting information from data sources specified by a user, analyzing the data, and providing useful insights. The system includes a terminal unit, a data collection unit, a data preprocessing unit, a data analysis unit, a knowledge extraction unit, and an information provision unit.

[0520] System configuration and operation

[0521] Terminal means

[0522] The user uses the device interface to specify a particular data source (e.g., a social media platform, a news site) and associated hashtags, and this information is sent to the server in the form of a request.

[0523] Data collection methods

[0524] The server uses APIs to gather information from data sources specified by the user, for example, using the Twitter API or a news site's API to gather posts and articles related to a particular hashtag.

[0525] Data preprocessing measures

[0526] The server cleanses the collected data. Specifically, it removes unnecessary symbols and links from text data, standardizes the resolution of image data, and removes noise. It also performs noise reduction and segmentation on audio data. For example, it cleanses text data using the Python pandas library.

[0527] Data Analysis Methods

[0528] The cleansed data is then analyzed by the server. Natural language processing (NLP) is used to perform sentiment analysis on the text data, and image recognition is used to identify specific objects or scenes within images. For audio data, a speech recognition algorithm is used to convert it into text and analyze its content. Specifically, Google's BERT model is used for natural language processing, and OpenCV is used for image recognition.

[0529] Knowledge extraction means

[0530] From the analysis results, the server identifies correlations and specific patterns between different data sources and extracts key insights, such as the correlation between sentiment scores of social media posts and stock prices in market data. This process is powered by the Scikit-learn library.

[0531] Information provision means

[0532] The extracted insights are delivered in a customized format based on user preferences and historical trends, visualized on a dashboard, or sent to users as periodic reports. For example, the dashboards are generated using Tableau, and the reports are generated using Python's ReportLab.

[0533] Specific examples

[0534] If a user wants to analyze the market valuation of a new product, they would follow these steps:

[0535] 1. Users enter the name of a new product and related hashtags into the system and specify Twitter or a news site as the data source.

[0536] 2. The server collects relevant information from these data sources using the specified APIs.

[0537] 3. After the collected data is cleansed, sentiment analysis is performed using natural language processing. For example, Twitter posts are analyzed using the BERT model to classify them as positive, negative, or neutral.

[0538] 4. The server uses these analysis results to extract market trends and patterns of product reviews. It then creates graphs to visualize periods of time when positive reviews spike and reactions in specific market segments. Matplotlib is used to generate these graphs.

[0539] 5. Extracted insights are sent to users in regular reports, with specific insights such as, "Positive posts about new products are most prevalent between 2:00 PM and 4:00 PM."

[0540] Prompt Sentence Examples

[0541] I'd like to analyze the market evaluation of new product XYZ, so I'd like you to collect posts related to XYZ from Twitter and news sites and perform a sentiment analysis. For example, I'd like to know how many positive and negative posts there are.

[0542] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, making it possible to quickly provide useful insights to users.

[0543] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0544] Step 1:

[0545] User request submission

[0546] The user uses the device interface to specify a specific data source (e.g., a social media platform, a news site) and relevant hashtags. Once the user enters this information and presses the send button, the request is sent from the device to the server.

[0547] Input: Data source and hashtags entered by the user into the form (e.g., "Twitter" or "New product name")

[0548] Output: Request data from the terminal to the server

[0549] Specific behavior:

[0550] A user enters the name of a new product and related hashtags into a form.

[0551] The user selects Twitter or a news site.

[0552] The user presses the "Send" button.

[0553] The terminal transmits the request data to the server.

[0554] Step 2:

[0555] Data collection by the server

[0556] Based on the request received, the server uses APIs to gather information from specified data sources, for example, Twitter API or a news site's API to retrieve posts and articles related to a specific hashtag.

[0557] Input: Request data sent from the terminal

[0558] Output: Raw data obtained from each data source

[0559] Specific behavior:

[0560] The server accesses the Twitter API and collects posts containing "new product name."

[0561] The server accesses the news site's API and collects articles related to the keyword "new product name."

[0562] The server stores this raw data internally.

[0563] Step 3:

[0564] Data preprocessing by the server

[0565] The server cleanses the collected data by removing unnecessary symbols and links from text data, standardizing the resolution of image data and removing noise, and performing noise reduction and segmentation on audio data.

[0566] Input: Raw data obtained from each data source

[0567] Output: Cleansed data

[0568] Specific behavior:

[0569] The server removes unnecessary links and symbols from the Twitter posts it retrieves.

[0570] The server extracts the body of the news article and removes any advertisements or sidebar information.

[0571] The server standardizes the resolution of the image data and removes noise.

[0572] Step 4:

[0573] Data analysis by server

[0574] The server analyzes the pre-processed data: natural language processing (NLP) is used to perform sentiment analysis on text data, image recognition is used to identify specific objects on image data, and speech recognition algorithms are used to convert audio data into text and analyze its content.

[0575] Input: Cleansed data

[0576] Output: Analysis result data

[0577] Specific behavior:

[0578] The server performs sentiment analysis on the Twitter posts retrieved by the server using the BERT model, classifying them as positive, negative, or neutral.

[0579] Analyze the titles and text of news articles to extract trending topics and keywords.

[0580] The server performs object recognition on the image data using OpenCV.

[0581] The audio data is converted into text and the content is further analyzed.

[0582] Step 5:

[0583] Knowledge extraction by server

[0584] The server extracts key findings from the data analysis, finding correlations and specific patterns between different data sources and generating useful insights.

[0585] Input: Analysis result data

[0586] Output: Knowledge data

[0587] Specific behavior:

[0588] The server compares the Twitter sentiment analysis results with market stock price data to find correlations between the two.

[0589] The increase or decrease in positive posts during specific time periods is graphed, suggesting applications for marketing.

[0590] Step 6:

[0591] Information provided by the server

[0592] The server provides the extracted insights to the user, visualizing them on a dashboard or sending them to the user as regular reports. Specifically, it generates graphs and tables to visualize the insights and provides the insights in a format that is easy for the user to understand.

[0593] Input: Knowledge data

[0594] Output: Customization information provided to the user

[0595] Specific behavior:

[0596] The server generates a graph of times when there are many positive posts and displays it on the dashboard.

[0597] Generate a weekly report in PDF format and send it to your email address.

[0598] In this way, the system can perform clear steps and provide useful insights to the user quickly.

[0599] (Application example 1)

[0600] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] Conventional surveillance systems have difficulty in monitoring a wide area or detecting suspicious individuals or objects in real time. Furthermore, there have been no systems that provide risk information by integrating information collection from social media and sentiment analysis. This has made it difficult to make quick and accurate decisions, and has led to problems in implementing effective security measures.

[0602] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0603] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, a video recognition means for monitoring video data in real time and detecting suspicious objects and people, and a natural language processing means for collecting risk information from social media and performing sentiment analysis. This enables wide-area monitoring and real-time detection of suspicious people and objects, and also enables risk information to be provided by integrating social media information, enabling quick and accurate decision-making.

[0604] A "data collection method" is a method by which a user collects information from a particular data source.

[0605] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary information, and preparing the data in an appropriate format.

[0606] "Data analysis means" refers to means for analyzing cleansed data to extract specific patterns and trends.

[0607] "Knowledge extraction means" refers to a means for extracting important knowledge from the results of data analysis.

[0608] "Information provision means" refers to a means for providing extracted knowledge to users.

[0609] "Video recognition means" is a means for monitoring video data in real time and detecting suspicious objects and people.

[0610] "Natural language processing means" is a means for collecting risk information from social media and performing sentiment analysis.

[0611] The present invention aims to realize a security service that collects information from data sources specified by a user, analyzes the information, and provides useful insights. The system includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means.

[0612] Data collection methods:

[0613] The server collects real-time video from devices such as surveillance cameras and drones. This video data is used to detect suspicious objects and people. Additionally, posts related to designated hashtags on social media (e.g., Twitter) can be collected via API. This allows users to efficiently collect a wide range of information.

[0614] Data preprocessing methods:

[0615] The server cleanses the collected video and social media data. For video data, it standardizes image quality and removes noise, and for text data, it removes unnecessary symbols and links. This improves the accuracy of the data and increases the efficiency of subsequent analysis.

[0616] Data analysis methods:

[0617] The server applies video recognition algorithms and natural language processing (NLP) to the cleansed data. Video recognition is used to detect suspicious objects and people in real time from surveillance camera footage. NLP is also used to perform sentiment analysis of social media posts and extract potential risk information. These analyses are performed using existing libraries such as OpenCV and TextBlob.

[0618] Knowledge extraction means:

[0619] The server extracts important insights from the analysis results. For example, it integrates location information of suspicious objects detected from video data and risk information extracted from social media, and finds correlations. This allows users to make accurate decisions in real time.

[0620] Information provision method:

[0621] The server uses a head-mounted display (HMD) to provide extracted insights to users, showing the location of suspicious objects and people in real time and providing social media sentiment analysis results as warnings, allowing users to take prompt and appropriate action.

[0622] Examples:

[0623] For example, if a security camera monitors an area in real time and detects a suspicious individual, this information will be sent to security guards via the HMD. Meanwhile, if social media posts related to the same area are analyzed and any disturbing information is found, this information will also be sent to security guards. This allows security guards to grasp the overall situation and respond quickly.

[0624] Example prompt sentence:

[0625] Below is an example of a prompt for a system using the present invention:

[0626] "I want to create an AI model that collects video data from surveillance cameras in real time and detects suspicious people and objects. I want to preprocess the collected video data and analyze it using the AI ​​model. How can I then build a system that displays the location information and warnings of detected suspicious objects in real time?"

[0627] "I want to collect posts related to a specified hashtag on Twitter and perform sentiment analysis. How can I build a system that extracts risk information in a specific area and issues real-time alerts to security guards?"

[0628] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0629] Step 1:

[0630] Data collection

[0631] The server acquires real-time video data from devices such as surveillance cameras and drones. The input is a video data source such as a surveillance camera, and the server captures and saves it frame by frame. In parallel, it uses the specified social media API to collect posts containing related hashtags and keywords. In this case, the input is the hashtag or keyword, and the output is the related SNS post data.

[0632] Step 2:

[0633] Data Preprocessing

[0634] The server cleanses the collected video data. Specific operations include unifying the resolution of the video data, removing noise, and improving image quality. The input is the video data collected in step 1, and the output is the preprocessed video data. For social media data, unnecessary symbols and links are deleted and text data is cleaned. The input is text data collected from SNS, and the output is preprocessed text data.

[0635] Step 3:

[0636] Data analysis

[0637] The server applies a video recognition algorithm to the preprocessed video data. Specifically, it performs object detection to detect suspicious objects and people for each frame. The input is the preprocessed video data, and the output is information about the detected suspicious objects and people. In addition, it performs sentiment analysis on the preprocessed text data using natural language processing (NLP). This allows it to obtain positive or negative sentiment scores as output from the processed text data as input.

[0638] Step 4:

[0639] Knowledge extraction

[0640] The server extracts important insights from the results of the data analysis. Specifically, it extracts location information of suspicious objects detected from video data and behavioral patterns of suspicious individuals. The input is the analysis results obtained in step 3, and the output is the extracted insights (detailed information on suspicious objects and individuals). Similarly, it extracts posts with high negative scores in specific regions from the results of social media sentiment analysis.

[0641] Step 5:

[0642] Providing information

[0643] The server uses a head-mounted display (HMD) to provide the extracted insights to the user. Specifically, it displays the location information of suspicious objects and people on the HMD in real time and issues warnings based on the results of social media sentiment analysis. The input is the insights extracted in step 4, and the output is the information and warnings displayed on the HMD. This allows the user to take prompt and appropriate action.

[0644] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0645] The present invention is a system that includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means, as well as an emotion engine that recognizes user emotions. This system has a mechanism for collecting information from data sources specified by the user, analyzing that data, and providing useful insights. It also has a function for customizing the information provided by taking the user's emotions into consideration.

[0646] Data collection methods:

[0647] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[0648] Data preprocessing methods:

[0649] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[0650] Data analysis methods:

[0651] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[0652] Knowledge extraction means:

[0653] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[0654] Information provision method:

[0655] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the information is customized based on the user's sentiment, and visualized on a dashboard or sent to the user as regular reports. Users can then make quick and accurate decisions based on this information.

[0656] Emotion Engine:

[0657] It has the ability to recognize the user's emotions in real time and adjust the information it provides based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that will reduce stress.

[0658] Specific examples

[0659] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[0660] For example, if the emotion engine detects that a user is feeling anxious, it will create a report centered around positive reviews and success stories, providing information that will reduce the user's stress, allowing the user to make decisions about marketing strategies and product improvements with peace of mind.

[0661] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and further customizes information taking into account the user's emotions, making it possible to quickly provide the most useful insights to the user.

[0662] The processing flow will be explained below.

[0663] Step 1: Specify the data source

[0664] Users tell the system the type of data they want to collect (e.g., text, images, audio) and the corresponding data source (e.g., social media platform, news site).

[0665] Step 2: Submit a request for data collection

[0666] The device sends a data collection request to the server based on the user's specifications, including the data source URL and query parameters.

[0667] Step 3: Collect data

[0668] The server calls the specified API (e.g., Twitter API, RSS feed) to collect data, which is then temporarily stored.

[0669] Step 4: Cleanse the data

[0670] The server cleanses the collected data.

[0671] For text data: Remove HTML tags, unnecessary symbols, and links and purify the data into text.

[0672] For image data: The resolution is unified and noise is removed.

[0673] For audio data: Apply a noise reduction filter to improve audio clarity.

[0674] Step 5: Data analysis

[0675] The server passes the cleansed data to various AI algorithms for analysis.

[0676] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction of text data.

[0677] Image recognition algorithms identify specific objects and scenes.

[0678] A speech recognition algorithm converts the voice data into text and then analyzes the content.

[0679] Step 6: Extracting insights

[0680] The server extracts key insights from the results of data analysis, which includes finding correlations between different data sources and pattern recognition.

[0681] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[0682] Step 7: Recognizing user emotions

[0683] The emotion engine analyzes the user's real-time emotional state, including emotion analysis from voice data and emotion inference from user input behavior.

[0684] Step 8: Customize your information offering

[0685] The server adjusts the information provided to the user based on the emotional state obtained by the emotion engine.

[0686] Example: If a user is in an anxious state, prioritize positive information.

[0687] Step 9: Provide information

[0688] The server provides tailored insights to users, including visualization in a dashboard and the generation and delivery of periodic reports.

[0689] Information is updated in real time on the dashboard and displayed visually in an easy-to-understand manner.

[0690] The analysis results will be emailed to the user as a weekly report.

[0691] Step 10: User feedback

[0692] Users make decisions based on the information provided and input their results and feedback into the system, which allows the system to learn and reflect this in the next information provided.

[0693] The above is the specific processing flow in the system of the present invention. In this way, it becomes possible to provide useful insights to users quickly and accurately.

[0694] Example 2

[0695] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0696] Conventional data analysis systems process and provide collected data in a uniform manner, which means they are unable to consider the emotional state or individual needs of users and are unable to support optimal decision-making.In addition, the wide variety of data sources makes it difficult to efficiently cleanse and analyze collected data.

[0697] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0698] In this invention, the server includes a data collection means, a preprocessing means for cleansing the collected data, an analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing the user's emotions and adjusting information based on that state. This makes it possible to provide information customized according to the user's emotional state, thereby supporting faster and more effective decision-making.

[0699] "Data Collection Implement" means any device or software used to collect information from user-specified data sources.

[0700] A "pre-processing means" is a device or software that cleanses the collected data and prepares it in a form suitable for analysis.

[0701] "Analysis means" refers to a device or software that analyzes the cleansed data using various algorithms to extract useful information and patterns.

[0702] The "knowledge extraction means" is a device or software for extracting important insights and meaningful knowledge from the analysis results obtained by the analysis means.

[0703] "Information provision means" refers to devices or software that provide extracted knowledge to users in an easy-to-understand manner. Specifically, this includes visualization on a dashboard and generation of reports.

[0704] An "emotion engine" is a device or software that recognizes a user's emotional state in real time and adjusts the information provided based on that state.

[0705] "API" stands for Application Programming Interface, a standardized way of sending and receiving data between different software applications.

[0706] "Natural language processing" is the technology for understanding and analyzing human language, and is used for sentiment analysis and semantic analysis of text.

[0707] "Image recognition" is the technology of identifying specific objects or patterns from image data.

[0708] "Speech recognition" is a technology that converts voice data into text and analyzes its content.

[0709] The present invention is a system that includes a data collection means, a preprocessing means, an analysis means, a knowledge extraction means, an information provision means, and an emotion engine. This allows the system to collect information from a data source specified by a user, analyze the data, and provide useful knowledge. The system also has a function to adjust the information taking the user's emotions into account.

[0710] Data collection methods

[0711] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies a specific hashtag, the server retrieves posts related to that hashtag. This collected data is then stored in the server's internal database.

[0712] Pretreatment means

[0713] The collected data is cleansed by the server. In the case of text data, unnecessary symbols and links are removed and the data is formatted appropriately. In the case of image data, the resolution is standardized and noise is removed. In the case of audio data, noise reduction and segmentation are performed. This preprocessing converts the data into a format suitable for analysis.

[0714] analytical means

[0715] The cleansed data is then analyzed by the server using various AI algorithms. Specifically, natural language processing (NLP) is used to perform sentiment analysis on text data, and image recognition is used to identify specific objects or scenes in images. Voice data is converted into text using speech recognition algorithms, and its content is analyzed. These algorithms are implemented using generative AI models.

[0716] Knowledge extraction means

[0717] The server extracts key insights from the analysis results, for example, by finding correlations between different data sources and analyzing the correlation between sentiment scores of social media posts and stock prices from market data, allowing users to gain deeper insights.

[0718] Information provision means

[0719] The extracted insights are delivered in a customized format based on the user's settings and past trends. The server visualizes these insights on a dashboard or sends them to the user as regular reports, allowing the user to make quick and accurate decisions based on the information provided.

[0720] Emotion Engine

[0721] The emotion engine recognizes the user's emotions in real time and adjusts the information provided based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress. This allows the system to provide the most appropriate information for the user.

[0722] Specific examples

[0723] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies social media platforms and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[0724] Prompt Sentence Examples

[0725] For example, a user might prompt a generative AI model as follows:

[0726] "I want to analyze the market evaluation of a new product using social media platforms and news sites as data sources. The related hashtag is new product 2023. Based on the results of the sentiment engine, please create a report with positive evaluations and success stories."

[0727] In this way, by automating the entire process from data collection to providing insights, and further customizing information taking into account user emotions, we have created a system that can quickly provide the most useful insights to users.

[0728] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0729] Step 1:

[0730] The user specifies a particular data source.

[0731] Users enter the required keywords or hashtags into the system's interface.

[0732] For example, a user enters the hashtag "New Products 2023."

[0733] Input data: hashtag "New Products 2023"

[0734] Output: API request data as a user request

[0735] Specific operation: When the user clicks the search button, the device converts the entered data into an API request format and sends it to the server.

[0736] Step 2:

[0737] The terminal sends the user's request to the server.

[0738] The device converts the request, including the hashtags and keywords entered by the user, into JSON format.

[0739] Input data: User-entered keywords (API request data)

[0740] Output: API request sent to the server

[0741] Specific operation: The device sends an API request to the server in the form of an HTTP request.

[0742] Step 3:

[0743] The server uses the specified API to collect information from the data source.

[0744] The server sends a request to the API endpoint of the social media platform or news site to retrieve the required data.

[0745] Input data: API request

[0746] Output: Data collected from social media platforms and news sites (text, images, audio, etc.)

[0747] Specific operation: The server sends a request to the API endpoint and stores the returned JSON data in an internal database.

[0748] Step 4:

[0749] The server cleanses the collected data.

[0750] The server removes unnecessary symbols and links from the text data and formats it appropriately. For image data, it standardizes the resolution and removes noise. For audio data, it performs noise reduction and segmentation.

[0751] Input data: raw data collected

[0752] Output: Cleansed data (clean text, images, audio data)

[0753] Specific operation: The server executes the data cleansing algorithm, filtering and format conversion.

[0754] Step 5:

[0755] The server analyzes the cleansed data.

[0756] The server uses natural language processing (NLP) to perform sentiment analysis on the text data, image recognition to identify specific objects and scenes in images, and speech recognition algorithms to convert audio data into text and analyze it.

[0757] Input data: Cleansed data (text, image, audio data)

[0758] Output: Analysis results (emotion scores, recognition results, text conversion data, etc.)

[0759] Specific operation: The server runs NLP algorithms and image recognition algorithms to generate emotion scores and object identification results.

[0760] Step 6:

[0761] The server extracts key insights from the analysis results.

[0762] The server analyzes correlations between different data sources and extracts key insights and findings.

[0763] Input data: Analysis results

[0764] Output: Extracted findings (correlations, patterns, insights, etc.)

[0765] What it does: The server analyzes the analysis results in the database and runs algorithms to extract key insights.

[0766] Step 7:

[0767] The server customizes the insights to provide to the user.

[0768] The server customizes the extracted insights based on the user's past preferences and tendencies.

[0769] Input data: extracted knowledge, user setting data

[0770] Output: Customized insights (reports and dashboards tailored to the user)

[0771] Specific operation: The server retrieves the user's profile data and uses the extracted insights to customize it.

[0772] Step 8:

[0773] The server uses an emotion engine to recognize the user's emotions in real time.

[0774] The emotion engine analyzes the user's emotional state from their facial expressions and voice.

[0775] Input data: Real-time facial expression data and voice data of the user

[0776] Output: User's emotional state (positive, negative, etc.)

[0777] Specific operation: The emotion engine runs facial expression recognition algorithms and voice analysis algorithms to evaluate the user's emotions.

[0778] Step 9:

[0779] The server tailors the information based on the user's emotional state.

[0780] The server adjusts the content of the information it provides depending on the recognized emotional state.

[0781] Input data: user's emotional state, customized insights

[0782] Output: Emotionally adjusted information (reports and feedback containing positive information)

[0783] Specific operation: The server filters information based on the emotional state, selects appropriate information and provides it to the user.

[0784] Step 10:

[0785] The server provides the information to the user.

[0786] The server visualizes the information on a dashboard or sends it to the user as periodic reports.

[0787] Input data: emotion-modulated information

[0788] Output: The final information provided to the user (dashboard display, report sending, etc.)

[0789] What it does: The server generates graphs and tables to visualize the information and displays them on the user's dashboard. It also generates a report in PDF format and emails it to the user.

[0790] This will automate the entire process from data collection to providing insights, and by customizing information taking into account user emotions, a system will be created that supports quick and accurate decision-making.

[0791] (Application example 2)

[0792] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0793] Conventional data collection and analysis systems are unable to provide information that takes into account the user's emotions, which can result in users receiving inappropriate information. Furthermore, there is no system in place to recognize customer emotions in real time and respond appropriately based on those emotions, making it difficult for store staff to respond quickly and accurately to customer needs.

[0794] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0795] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing customer emotions in real time and customizing the information to be provided. This makes it possible to provide the most appropriate information to the user and to present an appropriate response method according to the customer's emotions.

[0796] 1. "Data Collection Implement" means a device or system capable of collecting information from user-specified data sources.

[0797] 2. "Data preprocessing means" refers to a device or system that removes unnecessary information and cleanses data in order to prepare collected data in an appropriate format.

[0798] 3. "Data Analysis Tool" means a device or system that uses AI algorithms to analyze cleansed data and extract useful insights.

[0799] 4. A "Insight Extraction Tool" is a device or system used to find significant patterns and correlations from the results of data analysis.

[0800] 5. "Information Delivery Measure" means a device or system that delivers extracted insights to a user in a customized format based on the user's preferences and past trends.

[0801] 6. An "emotion engine" is a device or system that has the ability to recognize the emotions of users or customers in real time and adjust the information it provides based on that state.

[0802] 7. "API" means an interface for exchanging functions and data between different software programs, and is used to collect information from specified data sources.

[0803] 8. "Natural language processing" is a technology for analyzing text data and performing sentiment analysis and understanding intent.

[0804] 9. “Image recognition” is a technology for identifying specific objects or scenes from image data.

[0805] 10. "Speech recognition" is a technology for converting voice data into text and analyzing its content.

[0806] The system of the present invention mainly includes the following means: data collection means, data preprocessing means, data analysis means, knowledge extraction means, information provision means, and an emotion engine. This system has a mechanism for collecting information from data sources specified by the user, analyzing the data, and providing useful insights. It also has a function for recognizing the emotions of users and customers in real time and customizing the information provided based on those emotions.

[0807] First, when a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the necessary information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[0808] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[0809] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using speech recognition algorithms, and its content is then further analyzed.

[0810] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[0811] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the emotion engine recognizes the user's and customer's emotions in real time and adjusts the information provided based on their state. For example, if a user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress.

[0812] A specific example is when store staff use their smartphones to capture customers' facial expressions and tone of voice while serving them, and then suggest appropriate ways to respond based on the analysis results. This allows staff to suggest products and provide services that are appropriate to the customer's emotions.

[0813] Example prompt sentence:

[0814] "Identify the specific emotion this customer is experiencing and, based on that, suggest the best product or customer service approach."

[0815] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0816] Step 1:

[0817] The user specifies a specific data source (e.g., a social media platform or a news site) and sends a request to the device.

[0818] Input: User-specified data source and associated parameters (e.g., hashtags)

[0819] Output: Request data from the terminal to the server

[0820] How it works: The user interacts with the application and inputs the data source to be investigated and related parameters. The device then sends this information to the server.

[0821] Step 2:

[0822] The server uses the API to collect information from the specified data sources.

[0823] Input: Request data sent from the terminal

[0824] Output: Raw data retrieved from the specified data source.

[0825] What it does: The server makes an API request to gather information from the specified data source (e.g., Twitter API), including related posts and articles.

[0826] Step 3:

[0827] The server cleanses the collected data.

[0828] Input: Raw data collected

[0829] Output: Cleansed data

[0830] How it works: The server removes unnecessary symbols and links from text data, standardizes the resolution of image data, and reduces noise in audio data, thereby standardizing the data.

[0831] Step 4:

[0832] The server analyzes the cleansed data.

[0833] Input: Cleansed data

[0834] Output: Analysis results (e.g., sentiment scores, identified objects)

[0835] How it works: The server uses natural language processing (NLP) algorithms to perform sentiment analysis on text data, image data to analyze using image recognition algorithms, and audio data to convert it into text using speech recognition algorithms, which then analyzes its content.

[0836] Step 5:

[0837] The server extracts important insights from the analysis results.

[0838] Input: Analysis results

[0839] Output: Key findings (e.g. correlations, specific patterns)

[0840] How it works: The server finds correlations between different data sources and identifies specific patterns, such as the correlation between sentiment scores from social media posts and stock prices from market data.

[0841] Step 6:

[0842] The server provides the extracted knowledge to the user.

[0843] Input: Key Findings

[0844] Output: Customized information (e.g. reports, dashboards)

[0845] How it works: The server customizes insights based on user preferences and historical trends, visualizes them on a dashboard, or sends them to the user as regular reports.

[0846] Step 7:

[0847] The server uses an emotion engine to recognize the user's emotions in real time and adjusts information based on that state.

[0848] Input: User emotion data (e.g., facial expressions, tone of voice)

[0849] Output: Tailored information

[0850] How it works: The server uses a camera and microphone to capture and analyze the customer's facial expressions and tone of voice. Based on the analysis results, it provides information that corresponds to the user's emotional state. For example, if the user is anxious, it prioritizes positive information.

[0851] Step 8:

[0852] Users and store staff receive the information provided by the server and apply it to their actual work.

[0853] Input: Customized information provided by the server

[0854] Output: Real-world decisions and actions

[0855] How it works: Users and store staff decide business policies based on reports and dashboards provided by the server, and reflect these in customer service and marketing strategies.

[0856] This allows users to receive relevant insights in a timely manner, enabling them to provide better services and make better decisions based on customer sentiment.

[0857] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0858] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0859] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0860] [Third embodiment]

[0861] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0862] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0863] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0864] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0865] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0866] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0867] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0868] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0869] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0870] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0871] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0872] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0873] The present invention is a system including a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means. This system has a mechanism for collecting information from data sources specified by a user, analyzing the data, and providing useful insights.

[0874] Data collection methods:

[0875] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[0876] Data preprocessing methods:

[0877] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[0878] Data analysis methods:

[0879] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[0880] Knowledge extraction means:

[0881] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[0882] Information provision method:

[0883] Extracted insights are delivered to users in a customized format based on their preferences and historical trends, visualized on a dashboard, or sent to them as regular reports, allowing them to make quick and accurate decisions.

[0884] Specific examples

[0885] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server then uses the specified API to collect relevant information from these data sources. The collected data is then cleansed and sentiment analyzed using natural language processing. As a result, associations with times when there are many positive posts and specific keywords are discovered.

[0886] The server then uses these analyses to extract market trends and patterns in product reviews, such as graphs showing spikes in positive reviews or responses in specific market segments. This information is then sent to users in regular reports, allowing them to make decisions about marketing strategies and product improvements.

[0887] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and is able to quickly provide useful insights to users.

[0888] The processing flow will be explained below.

[0889] Step 1: User-specified data source configuration

[0890] Users tell the system the type of data they want to collect and the data source (e.g., social media platforms or news sites).

[0891] Step 2: Request to start data collection

[0892] The device sends a data collection request to the server based on the user's settings.

[0893] Step 3: Data Acquisition

[0894] The server calls the specified API (e.g., Twitter API, RSS feed) and collects the data. At this time, the URL and query parameters to be accessed are dynamically generated according to the user's settings.

[0895] Example: Collect related posts from the Twitter API based on the user-specified hashtag "market trends."

[0896] Step 4: Data cleansing

[0897] The server cleanses the collected data.

[0898] For text data: Remove HTML tags, unnecessary symbols, and links.

[0899] For image data: unify the resolution and reduce noise.

[0900] For audio data: Apply a noise reduction filter to improve audio clarity.

[0901] Step 5: Analytical processing according to the data format

[0902] The server passes the cleansed data to various AI algorithms for analysis.

[0903] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction from text data.

[0904] Image recognition algorithms are used to recognize specific objects and scenes.

[0905] Using a speech recognition algorithm, the voice data is converted into text and further subjected to sentiment analysis.

[0906] Step 6: Find data correlations

[0907] The server analyzes the relationships between different data sources and extracts important correlations.

[0908] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[0909] Step 7: Creating a visualization

[0910] The server generates dashboards and graphs to visually illustrate the extracted correlations.

[0911] Example: Graphing sentiment analysis results on a timeline.

[0912] Step 8: Generate reports for users

[0913] The server compiles insights from user settings and historical data to create customized reports.

[0914] Example: Generate a weekly report summarizing market trends and sentiment analysis related to a specified hashtag.

[0915] Step 9: Notification and distribution

[0916] The device will notify the user of important announcements and analysis results at specific times.

[0917] Example: Sending reports generated by the server to users via email.

[0918] Example: Analytical results are updated in real time on a dashboard, providing users with a view into the results.

[0919] Example 1

[0920] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0921] Conventional data analysis systems have struggled to provide users with the information they need quickly and accurately, particularly in the complex and time-consuming process of consistently preprocessing data collected from different data sources and extracting useful insights in real time to provide them to users.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0923] In this invention, the server includes a data collection means for collecting information from specified data sources using APIs, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data using natural language processing, image recognition, and voice recognition, a knowledge extraction means for identifying correlations between different data sources and specific patterns from the analysis results and extracting important knowledge, and an information provision means for providing the extracted knowledge in a customized format based on user settings and past trends, visualizing it on a dashboard, or sending it to the user as a regular report, thereby enabling the user to quickly and accurately obtain useful insights.

[0924] Below are definitions of key words.

[0925] The "terminal means" is a device for inputting a data source and a request designated by a user and transmitting the input to a server.

[0926] "Data collection means" means a means of obtaining necessary information from a specified data source using an API.

[0927] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary symbols and links, and arranging the data into an appropriate format.

[0928] "Data analysis means" refers to means for analyzing cleansed data using natural language processing, image recognition, and voice recognition.

[0929] "Insight extraction means" refers to means for identifying correlations and specific patterns between different data sources from the results of data analysis and extracting important insights.

[0930] "Information provision means" refers to the means of providing extracted knowledge to users, visualizing it on a dashboard, or sending it as a regular report.

[0931] "API" is an abbreviation for Application Programming Interface, an interface for exchanging data between different software programs.

[0932] "Natural language processing" is a technology that uses computers to understand and process human language.

[0933] "Image recognition" is a technology that identifies specific objects from digital images and videos.

[0934] "Speech recognition" is a technology that converts voice data into text and understands its content.

[0935] A "dashboard" is an interface that provides users with visualized data information.

[0936] A "report" is an information document that summarizes analysis results and findings in written form and provides them to users.

[0937] The present invention provides a system for collecting information from data sources specified by a user, analyzing the data, and providing useful insights. The system includes a terminal unit, a data collection unit, a data preprocessing unit, a data analysis unit, a knowledge extraction unit, and an information provision unit.

[0938] System configuration and operation

[0939] Terminal means

[0940] The user uses the device interface to specify a particular data source (e.g., a social media platform, a news site) and associated hashtags, and this information is sent to the server in the form of a request.

[0941] Data collection methods

[0942] The server uses APIs to gather information from data sources specified by the user, for example, using the Twitter API or a news site's API to gather posts and articles related to a particular hashtag.

[0943] Data preprocessing measures

[0944] The server cleanses the collected data. Specifically, it removes unnecessary symbols and links from text data, standardizes the resolution of image data, and removes noise. It also performs noise reduction and segmentation on audio data. For example, it cleanses text data using the Python pandas library.

[0945] Data Analysis Methods

[0946] The cleansed data is then analyzed by the server. Natural language processing (NLP) is used to perform sentiment analysis on the text data, and image recognition is used to identify specific objects or scenes within images. For audio data, a speech recognition algorithm is used to convert it into text and analyze its content. Specifically, Google's BERT model is used for natural language processing, and OpenCV is used for image recognition.

[0947] Knowledge extraction means

[0948] From the analysis results, the server identifies correlations and specific patterns between different data sources and extracts key insights, such as the correlation between sentiment scores of social media posts and stock prices in market data. This process is powered by the Scikit-learn library.

[0949] Information provision means

[0950] The extracted insights are delivered in a customized format based on user preferences and historical trends, visualized on a dashboard, or sent to users as periodic reports. For example, the dashboards are generated using Tableau, and the reports are generated using Python's ReportLab.

[0951] Specific examples

[0952] If a user wants to analyze the market valuation of a new product, they would follow these steps:

[0953] 1. Users enter the name of a new product and related hashtags into the system and specify Twitter or a news site as the data source.

[0954] 2. The server collects relevant information from these data sources using the specified APIs.

[0955] 3. After the collected data is cleansed, sentiment analysis is performed using natural language processing. For example, Twitter posts are analyzed using the BERT model to classify them as positive, negative, or neutral.

[0956] 4. The server uses these analysis results to extract market trends and patterns of product reviews. It then creates graphs to visualize periods of time when positive reviews spike and reactions in specific market segments. Matplotlib is used to generate these graphs.

[0957] 5. Extracted insights are sent to users in regular reports, with specific insights such as, "Positive posts about new products are most prevalent between 2:00 PM and 4:00 PM."

[0958] Prompt Sentence Examples

[0959] I'd like to analyze the market evaluation of new product XYZ, so I'd like you to collect posts related to XYZ from Twitter and news sites and perform a sentiment analysis. For example, I'd like to know how many positive and negative posts there are.

[0960] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, making it possible to quickly provide useful insights to users.

[0961] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0962] Step 1:

[0963] User request submission

[0964] The user uses the device interface to specify a specific data source (e.g., a social media platform, a news site) and relevant hashtags. Once the user enters this information and presses the send button, the request is sent from the device to the server.

[0965] Input: Data source and hashtags entered by the user into the form (e.g., "Twitter" or "New product name")

[0966] Output: Request data from the terminal to the server

[0967] Specific behavior:

[0968] A user enters the name of a new product and related hashtags into a form.

[0969] The user selects Twitter or a news site.

[0970] The user presses the "Send" button.

[0971] The terminal transmits the request data to the server.

[0972] Step 2:

[0973] Data collection by the server

[0974] Based on the request received, the server uses APIs to gather information from specified data sources, for example, Twitter API or a news site's API to retrieve posts and articles related to a specific hashtag.

[0975] Input: Request data sent from the terminal

[0976] Output: Raw data obtained from each data source

[0977] Specific behavior:

[0978] The server accesses the Twitter API and collects posts containing "new product name."

[0979] The server accesses the news site's API and collects articles related to the keyword "new product name."

[0980] The server stores this raw data internally.

[0981] Step 3:

[0982] Data preprocessing by the server

[0983] The server cleanses the collected data by removing unnecessary symbols and links from text data, standardizing the resolution of image data and removing noise, and performing noise reduction and segmentation on audio data.

[0984] Input: Raw data obtained from each data source

[0985] Output: Cleansed data

[0986] Specific behavior:

[0987] The server removes unnecessary links and symbols from the Twitter posts it retrieves.

[0988] The server extracts the body of the news article and removes any advertisements or sidebar information.

[0989] The server standardizes the resolution of the image data and removes noise.

[0990] Step 4:

[0991] Data analysis by server

[0992] The server analyzes the pre-processed data: natural language processing (NLP) is used to perform sentiment analysis on text data, image recognition is used to identify specific objects on image data, and speech recognition algorithms are used to convert audio data into text and analyze its content.

[0993] Input: Cleansed data

[0994] Output: Analysis result data

[0995] Specific behavior:

[0996] The server performs sentiment analysis on the Twitter posts retrieved by the server using the BERT model, classifying them as positive, negative, or neutral.

[0997] Analyze the titles and text of news articles to extract trending topics and keywords.

[0998] The server performs object recognition on the image data using OpenCV.

[0999] The audio data is converted into text and the content is further analyzed.

[1000] Step 5:

[1001] Knowledge extraction by server

[1002] The server extracts key findings from the data analysis, finding correlations and specific patterns between different data sources and generating useful insights.

[1003] Input: Analysis result data

[1004] Output: Knowledge data

[1005] Specific behavior:

[1006] The server compares the Twitter sentiment analysis results with market stock price data to find correlations between the two.

[1007] The increase or decrease in positive posts during specific time periods is graphed, suggesting applications for marketing.

[1008] Step 6:

[1009] Information provided by the server

[1010] The server provides the extracted insights to the user, visualizing them on a dashboard or sending them to the user as regular reports. Specifically, it generates graphs and tables to visualize the insights and provides the insights in a format that is easy for the user to understand.

[1011] Input: Knowledge data

[1012] Output: Customization information provided to the user

[1013] Specific behavior:

[1014] The server generates a graph of times when there are many positive posts and displays it on the dashboard.

[1015] Generate a weekly report in PDF format and send it to your email address.

[1016] In this way, the system can perform clear steps and provide useful insights to the user quickly.

[1017] (Application example 1)

[1018] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1019] Conventional surveillance systems have difficulty in monitoring a wide area or detecting suspicious individuals or objects in real time. Furthermore, there have been no systems that provide risk information by integrating information collection from social media and sentiment analysis. This has made it difficult to make quick and accurate decisions, and has led to problems in implementing effective security measures.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1021] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, a video recognition means for monitoring video data in real time and detecting suspicious objects and people, and a natural language processing means for collecting risk information from social media and performing sentiment analysis. This enables wide-area monitoring and real-time detection of suspicious people and objects, and also enables risk information to be provided by integrating social media information, enabling quick and accurate decision-making.

[1022] A "data collection method" is a method by which a user collects information from a particular data source.

[1023] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary information, and preparing the data in an appropriate format.

[1024] "Data analysis means" refers to means for analyzing cleansed data to extract specific patterns and trends.

[1025] "Knowledge extraction means" refers to a means for extracting important knowledge from the results of data analysis.

[1026] "Information provision means" refers to a means for providing extracted knowledge to users.

[1027] "Video recognition means" is a means for monitoring video data in real time and detecting suspicious objects and people.

[1028] "Natural language processing means" is a means for collecting risk information from social media and performing sentiment analysis.

[1029] The present invention aims to realize a security service that collects information from data sources specified by a user, analyzes the information, and provides useful insights. The system includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means.

[1030] Data collection methods:

[1031] The server collects real-time video from devices such as surveillance cameras and drones. This video data is used to detect suspicious objects and people. Additionally, posts related to designated hashtags on social media (e.g., Twitter) can be collected via API. This allows users to efficiently collect a wide range of information.

[1032] Data preprocessing methods:

[1033] The server cleanses the collected video and social media data. For video data, it standardizes image quality and removes noise, and for text data, it removes unnecessary symbols and links. This improves the accuracy of the data and increases the efficiency of subsequent analysis.

[1034] Data analysis methods:

[1035] The server applies video recognition algorithms and natural language processing (NLP) to the cleansed data. Video recognition is used to detect suspicious objects and people in real time from surveillance camera footage. NLP is also used to perform sentiment analysis of social media posts and extract potential risk information. These analyses are performed using existing libraries such as OpenCV and TextBlob.

[1036] Knowledge extraction means:

[1037] The server extracts important insights from the analysis results. For example, it integrates location information of suspicious objects detected from video data and risk information extracted from social media, and finds correlations. This allows users to make accurate decisions in real time.

[1038] Information provision method:

[1039] The server uses a head-mounted display (HMD) to provide extracted insights to users, showing the location of suspicious objects and people in real time and providing social media sentiment analysis results as warnings, allowing users to take prompt and appropriate action.

[1040] Examples:

[1041] For example, if a security camera monitors an area in real time and detects a suspicious individual, this information will be sent to security guards via the HMD. Meanwhile, if social media posts related to the same area are analyzed and any disturbing information is found, this information will also be sent to security guards. This allows security guards to grasp the overall situation and respond quickly.

[1042] Example prompt sentence:

[1043] Below is an example of a prompt for a system using the present invention:

[1044] "I want to create an AI model that collects video data from surveillance cameras in real time and detects suspicious people and objects. I want to preprocess the collected video data and analyze it using the AI ​​model. How can I then build a system that displays the location information and warnings of detected suspicious objects in real time?"

[1045] "I want to collect posts related to a specified hashtag on Twitter and perform sentiment analysis. How can I build a system that extracts risk information in a specific area and issues real-time alerts to security guards?"

[1046] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1047] Step 1:

[1048] Data collection

[1049] The server acquires real-time video data from devices such as surveillance cameras and drones. The input is a video data source such as a surveillance camera, and the server captures and saves it frame by frame. In parallel, it uses the specified social media API to collect posts containing related hashtags and keywords. In this case, the input is the hashtag or keyword, and the output is the related SNS post data.

[1050] Step 2:

[1051] Data Preprocessing

[1052] The server cleanses the collected video data. Specific operations include unifying the resolution of the video data, removing noise, and improving image quality. The input is the video data collected in step 1, and the output is the preprocessed video data. For social media data, unnecessary symbols and links are deleted and text data is cleaned. The input is text data collected from SNS, and the output is preprocessed text data.

[1053] Step 3:

[1054] Data analysis

[1055] The server applies a video recognition algorithm to the preprocessed video data. Specifically, it performs object detection to detect suspicious objects and people for each frame. The input is the preprocessed video data, and the output is information about the detected suspicious objects and people. In addition, it performs sentiment analysis on the preprocessed text data using natural language processing (NLP). This allows it to obtain positive or negative sentiment scores as output from the processed text data as input.

[1056] Step 4:

[1057] Knowledge extraction

[1058] The server extracts important insights from the results of the data analysis. Specifically, it extracts location information of suspicious objects detected from video data and behavioral patterns of suspicious individuals. The input is the analysis results obtained in step 3, and the output is the extracted insights (detailed information on suspicious objects and individuals). Similarly, it extracts posts with high negative scores in specific regions from the results of social media sentiment analysis.

[1059] Step 5:

[1060] Providing information

[1061] The server uses a head-mounted display (HMD) to provide the extracted insights to the user. Specifically, it displays the location information of suspicious objects and people on the HMD in real time and issues warnings based on the results of social media sentiment analysis. The input is the insights extracted in step 4, and the output is the information and warnings displayed on the HMD. This allows the user to take prompt and appropriate action.

[1062] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1063] The present invention is a system that includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means, as well as an emotion engine that recognizes user emotions. This system has a mechanism for collecting information from data sources specified by the user, analyzing that data, and providing useful insights. It also has a function for customizing the information provided by taking the user's emotions into consideration.

[1064] Data collection methods:

[1065] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[1066] Data preprocessing methods:

[1067] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[1068] Data analysis methods:

[1069] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[1070] Knowledge extraction means:

[1071] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[1072] Information provision method:

[1073] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the information is customized based on the user's sentiment, and visualized on a dashboard or sent to the user as regular reports. Users can then make quick and accurate decisions based on this information.

[1074] Emotion Engine:

[1075] It has the ability to recognize the user's emotions in real time and adjust the information it provides based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that will reduce stress.

[1076] Specific examples

[1077] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[1078] For example, if the emotion engine detects that a user is feeling anxious, it will create a report centered around positive reviews and success stories, providing information that will reduce the user's stress, allowing the user to make decisions about marketing strategies and product improvements with peace of mind.

[1079] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and further customizes information taking into account the user's emotions, making it possible to quickly provide the most useful insights to the user.

[1080] The processing flow will be explained below.

[1081] Step 1: Specify the data source

[1082] Users tell the system the type of data they want to collect (e.g., text, images, audio) and the corresponding data source (e.g., social media platform, news site).

[1083] Step 2: Submit a request for data collection

[1084] The device sends a data collection request to the server based on the user's specifications, including the data source URL and query parameters.

[1085] Step 3: Collect data

[1086] The server calls the specified API (e.g., Twitter API, RSS feed) to collect data, which is then temporarily stored.

[1087] Step 4: Cleanse the data

[1088] The server cleanses the collected data.

[1089] For text data: Remove HTML tags, unnecessary symbols, and links and purify the data into text.

[1090] For image data: The resolution is unified and noise is removed.

[1091] For audio data: Apply a noise reduction filter to improve audio clarity.

[1092] Step 5: Data analysis

[1093] The server passes the cleansed data to various AI algorithms for analysis.

[1094] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction of text data.

[1095] Image recognition algorithms identify specific objects and scenes.

[1096] A speech recognition algorithm converts the voice data into text and then analyzes the content.

[1097] Step 6: Extracting insights

[1098] The server extracts key insights from the results of data analysis, which includes finding correlations between different data sources and pattern recognition.

[1099] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[1100] Step 7: Recognizing user emotions

[1101] The emotion engine analyzes the user's real-time emotional state, including emotion analysis from voice data and emotion inference from user input behavior.

[1102] Step 8: Customize your information offering

[1103] The server adjusts the information provided to the user based on the emotional state obtained by the emotion engine.

[1104] Example: If a user is in an anxious state, prioritize positive information.

[1105] Step 9: Provide information

[1106] The server provides tailored insights to users, including visualization in a dashboard and the generation and delivery of periodic reports.

[1107] Information is updated in real time on the dashboard and displayed visually in an easy-to-understand manner.

[1108] The analysis results will be emailed to the user as a weekly report.

[1109] Step 10: User feedback

[1110] Users make decisions based on the information provided and input their results and feedback into the system, which allows the system to learn and reflect this in the next information provided.

[1111] The above is the specific processing flow in the system of the present invention. In this way, it becomes possible to provide useful insights to users quickly and accurately.

[1112] Example 2

[1113] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1114] Conventional data analysis systems process and provide collected data in a uniform manner, which means they are unable to consider the emotional state or individual needs of users and are unable to support optimal decision-making.In addition, the wide variety of data sources makes it difficult to efficiently cleanse and analyze collected data.

[1115] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1116] In this invention, the server includes a data collection means, a preprocessing means for cleansing the collected data, an analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing the user's emotions and adjusting information based on that state. This makes it possible to provide information customized according to the user's emotional state, thereby supporting faster and more effective decision-making.

[1117] "Data Collection Implement" means any device or software used to collect information from user-specified data sources.

[1118] A "pre-processing means" is a device or software that cleanses the collected data and prepares it in a form suitable for analysis.

[1119] "Analysis means" refers to a device or software that analyzes the cleansed data using various algorithms to extract useful information and patterns.

[1120] The "knowledge extraction means" is a device or software for extracting important insights and meaningful knowledge from the analysis results obtained by the analysis means.

[1121] "Information provision means" refers to devices or software that provide extracted knowledge to users in an easy-to-understand manner. Specifically, this includes visualization on a dashboard and generation of reports.

[1122] An "emotion engine" is a device or software that recognizes a user's emotional state in real time and adjusts the information provided based on that state.

[1123] "API" stands for Application Programming Interface, a standardized way of sending and receiving data between different software applications.

[1124] "Natural language processing" is the technology for understanding and analyzing human language, and is used for sentiment analysis and semantic analysis of text.

[1125] "Image recognition" is the technology of identifying specific objects or patterns from image data.

[1126] "Speech recognition" is a technology that converts voice data into text and analyzes its content.

[1127] The present invention is a system that includes a data collection means, a preprocessing means, an analysis means, a knowledge extraction means, an information provision means, and an emotion engine. This allows the system to collect information from a data source specified by a user, analyze the data, and provide useful knowledge. The system also has a function to adjust the information taking the user's emotions into account.

[1128] Data collection methods

[1129] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies a specific hashtag, the server retrieves posts related to that hashtag. This collected data is then stored in the server's internal database.

[1130] Pretreatment means

[1131] The collected data is cleansed by the server. In the case of text data, unnecessary symbols and links are removed and the data is formatted appropriately. In the case of image data, the resolution is standardized and noise is removed. In the case of audio data, noise reduction and segmentation are performed. This preprocessing converts the data into a format suitable for analysis.

[1132] analytical means

[1133] The cleansed data is then analyzed by the server using various AI algorithms. Specifically, natural language processing (NLP) is used to perform sentiment analysis on text data, and image recognition is used to identify specific objects or scenes in images. Voice data is converted into text using speech recognition algorithms, and its content is analyzed. These algorithms are implemented using generative AI models.

[1134] Knowledge extraction means

[1135] The server extracts key insights from the analysis results, for example, by finding correlations between different data sources and analyzing the correlation between sentiment scores of social media posts and stock prices from market data, allowing users to gain deeper insights.

[1136] Information provision means

[1137] The extracted insights are delivered in a customized format based on the user's settings and past trends. The server visualizes these insights on a dashboard or sends them to the user as regular reports, allowing the user to make quick and accurate decisions based on the information provided.

[1138] Emotion Engine

[1139] The emotion engine recognizes the user's emotions in real time and adjusts the information provided based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress. This allows the system to provide the most appropriate information for the user.

[1140] Specific examples

[1141] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies social media platforms and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[1142] Prompt Sentence Examples

[1143] For example, a user might prompt a generative AI model as follows:

[1144] "I want to analyze the market evaluation of a new product using social media platforms and news sites as data sources. The related hashtag is new product 2023. Based on the results of the sentiment engine, please create a report with positive evaluations and success stories."

[1145] In this way, by automating the entire process from data collection to providing insights, and further customizing information taking into account user emotions, we have created a system that can quickly provide the most useful insights to users.

[1146] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1147] Step 1:

[1148] The user specifies a particular data source.

[1149] Users enter the required keywords or hashtags into the system's interface.

[1150] For example, a user enters the hashtag "New Products 2023."

[1151] Input data: hashtag "New Products 2023"

[1152] Output: API request data as a user request

[1153] Specific operation: When the user clicks the search button, the device converts the entered data into an API request format and sends it to the server.

[1154] Step 2:

[1155] The terminal sends the user's request to the server.

[1156] The device converts the request, including the hashtags and keywords entered by the user, into JSON format.

[1157] Input data: User-entered keywords (API request data)

[1158] Output: API request sent to the server

[1159] Specific operation: The device sends an API request to the server in the form of an HTTP request.

[1160] Step 3:

[1161] The server uses the specified API to collect information from the data source.

[1162] The server sends a request to the API endpoint of the social media platform or news site to retrieve the required data.

[1163] Input data: API request

[1164] Output: Data collected from social media platforms and news sites (text, images, audio, etc.)

[1165] Specific operation: The server sends a request to the API endpoint and stores the returned JSON data in an internal database.

[1166] Step 4:

[1167] The server cleanses the collected data.

[1168] The server removes unnecessary symbols and links from the text data and formats it appropriately. For image data, it standardizes the resolution and removes noise. For audio data, it performs noise reduction and segmentation.

[1169] Input data: raw data collected

[1170] Output: Cleansed data (clean text, images, audio data)

[1171] Specific operation: The server executes the data cleansing algorithm, filtering and format conversion.

[1172] Step 5:

[1173] The server analyzes the cleansed data.

[1174] The server uses natural language processing (NLP) to perform sentiment analysis on the text data, image recognition to identify specific objects and scenes in images, and speech recognition algorithms to convert audio data into text and analyze it.

[1175] Input data: Cleansed data (text, image, audio data)

[1176] Output: Analysis results (emotion scores, recognition results, text conversion data, etc.)

[1177] Specific operation: The server runs NLP algorithms and image recognition algorithms to generate emotion scores and object identification results.

[1178] Step 6:

[1179] The server extracts key insights from the analysis results.

[1180] The server analyzes correlations between different data sources and extracts key insights and findings.

[1181] Input data: Analysis results

[1182] Output: Extracted findings (correlations, patterns, insights, etc.)

[1183] What it does: The server analyzes the analysis results in the database and runs algorithms to extract key insights.

[1184] Step 7:

[1185] The server customizes the insights to provide to the user.

[1186] The server customizes the extracted insights based on the user's past preferences and tendencies.

[1187] Input data: extracted knowledge, user setting data

[1188] Output: Customized insights (reports and dashboards tailored to the user)

[1189] Specific operation: The server retrieves the user's profile data and uses the extracted insights to customize it.

[1190] Step 8:

[1191] The server uses an emotion engine to recognize the user's emotions in real time.

[1192] The emotion engine analyzes the user's emotional state from their facial expressions and voice.

[1193] Input data: Real-time facial expression data and voice data of the user

[1194] Output: User's emotional state (positive, negative, etc.)

[1195] Specific operation: The emotion engine runs facial expression recognition algorithms and voice analysis algorithms to evaluate the user's emotions.

[1196] Step 9:

[1197] The server tailors the information based on the user's emotional state.

[1198] The server adjusts the content of the information it provides depending on the recognized emotional state.

[1199] Input data: user's emotional state, customized insights

[1200] Output: Emotionally adjusted information (reports and feedback containing positive information)

[1201] Specific operation: The server filters information based on the emotional state, selects appropriate information and provides it to the user.

[1202] Step 10:

[1203] The server provides the information to the user.

[1204] The server visualizes the information on a dashboard or sends it to the user as periodic reports.

[1205] Input data: emotion-modulated information

[1206] Output: The final information provided to the user (dashboard display, report sending, etc.)

[1207] What it does: The server generates graphs and tables to visualize the information and displays them on the user's dashboard. It also generates a report in PDF format and emails it to the user.

[1208] This will automate the entire process from data collection to providing insights, and by customizing information taking into account user emotions, a system will be created that supports quick and accurate decision-making.

[1209] (Application example 2)

[1210] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1211] Conventional data collection and analysis systems are unable to provide information that takes into account the user's emotions, which can result in users receiving inappropriate information. Furthermore, there is no system in place to recognize customer emotions in real time and respond appropriately based on those emotions, making it difficult for store staff to respond quickly and accurately to customer needs.

[1212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1213] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing customer emotions in real time and customizing the information to be provided. This makes it possible to provide the most appropriate information to the user and to present an appropriate response method according to the customer's emotions.

[1214] 1. "Data Collection Implement" means a device or system capable of collecting information from user-specified data sources.

[1215] 2. "Data preprocessing means" refers to a device or system that removes unnecessary information and cleanses data in order to prepare collected data in an appropriate format.

[1216] 3. "Data Analysis Tool" means a device or system that uses AI algorithms to analyze cleansed data and extract useful insights.

[1217] 4. A "Insight Extraction Tool" is a device or system used to find significant patterns and correlations from the results of data analysis.

[1218] 5. "Information Delivery Measure" means a device or system that delivers extracted insights to a user in a customized format based on the user's preferences and past trends.

[1219] 6. An "emotion engine" is a device or system that has the ability to recognize the emotions of users or customers in real time and adjust the information it provides based on that state.

[1220] 7. "API" means an interface for exchanging functions and data between different software programs, and is used to collect information from specified data sources.

[1221] 8. "Natural language processing" is a technology for analyzing text data and performing sentiment analysis and understanding intent.

[1222] 9. “Image recognition” is a technology for identifying specific objects or scenes from image data.

[1223] 10. "Speech recognition" is a technology for converting voice data into text and analyzing its content.

[1224] The system of the present invention mainly includes the following means: data collection means, data preprocessing means, data analysis means, knowledge extraction means, information provision means, and an emotion engine. This system has a mechanism for collecting information from data sources specified by the user, analyzing the data, and providing useful insights. It also has a function for recognizing the emotions of users and customers in real time and customizing the information provided based on those emotions.

[1225] First, when a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the necessary information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[1226] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[1227] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using speech recognition algorithms, and its content is then further analyzed.

[1228] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[1229] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the emotion engine recognizes the user's and customer's emotions in real time and adjusts the information provided based on their state. For example, if a user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress.

[1230] A specific example is when store staff use their smartphones to capture customers' facial expressions and tone of voice while serving them, and then suggest appropriate ways to respond based on the analysis results. This allows staff to suggest products and provide services that are appropriate to the customer's emotions.

[1231] Example prompt sentence:

[1232] "Identify the specific emotion this customer is experiencing and, based on that, suggest the best product or customer service approach."

[1233] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1234] Step 1:

[1235] The user specifies a specific data source (e.g., a social media platform or a news site) and sends a request to the device.

[1236] Input: User-specified data source and associated parameters (e.g., hashtags)

[1237] Output: Request data from the terminal to the server

[1238] How it works: The user interacts with the application and inputs the data source to be investigated and related parameters. The device then sends this information to the server.

[1239] Step 2:

[1240] The server uses the API to collect information from the specified data sources.

[1241] Input: Request data sent from the terminal

[1242] Output: Raw data retrieved from the specified data source.

[1243] What it does: The server makes an API request to gather information from the specified data source (e.g., Twitter API), including related posts and articles.

[1244] Step 3:

[1245] The server cleanses the collected data.

[1246] Input: Raw data collected

[1247] Output: Cleansed data

[1248] How it works: The server removes unnecessary symbols and links from text data, standardizes the resolution of image data, and reduces noise in audio data, thereby standardizing the data.

[1249] Step 4:

[1250] The server analyzes the cleansed data.

[1251] Input: Cleansed data

[1252] Output: Analysis results (e.g., sentiment scores, identified objects)

[1253] How it works: The server uses natural language processing (NLP) algorithms to perform sentiment analysis on text data, image data to analyze using image recognition algorithms, and audio data to convert it into text using speech recognition algorithms, which then analyzes its content.

[1254] Step 5:

[1255] The server extracts important insights from the analysis results.

[1256] Input: Analysis results

[1257] Output: Key findings (e.g. correlations, specific patterns)

[1258] How it works: The server finds correlations between different data sources and identifies specific patterns, such as the correlation between sentiment scores from social media posts and stock prices from market data.

[1259] Step 6:

[1260] The server provides the extracted knowledge to the user.

[1261] Input: Key Findings

[1262] Output: Customized information (e.g. reports, dashboards)

[1263] How it works: The server customizes insights based on user preferences and historical trends, visualizes them on a dashboard, or sends them to the user as regular reports.

[1264] Step 7:

[1265] The server uses an emotion engine to recognize the user's emotions in real time and adjusts information based on that state.

[1266] Input: User emotion data (e.g., facial expressions, tone of voice)

[1267] Output: Tailored information

[1268] How it works: The server uses a camera and microphone to capture and analyze the customer's facial expressions and tone of voice. Based on the analysis results, it provides information that corresponds to the user's emotional state. For example, if the user is anxious, it prioritizes positive information.

[1269] Step 8:

[1270] Users and store staff receive the information provided by the server and apply it to their actual work.

[1271] Input: Customized information provided by the server

[1272] Output: Real-world decisions and actions

[1273] How it works: Users and store staff decide business policies based on reports and dashboards provided by the server, and reflect these in customer service and marketing strategies.

[1274] This allows users to receive relevant insights in a timely manner, enabling them to provide better services and make better decisions based on customer sentiment.

[1275] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1276] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1277] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1278] [Fourth embodiment]

[1279] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1280] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1281] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1282] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1283] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1284] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1285] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1286] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1287] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1288] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1289] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1290] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1291] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1292] The present invention is a system including a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means. This system has a mechanism for collecting information from data sources specified by a user, analyzing the data, and providing useful insights.

[1293] Data collection methods:

[1294] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[1295] Data preprocessing methods:

[1296] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[1297] Data analysis methods:

[1298] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[1299] Knowledge extraction means:

[1300] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[1301] Information provision method:

[1302] Extracted insights are delivered to users in a customized format based on their preferences and historical trends, visualized on a dashboard, or sent to them as regular reports, allowing them to make quick and accurate decisions.

[1303] Specific examples

[1304] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server then uses the specified API to collect relevant information from these data sources. The collected data is then cleansed and sentiment analyzed using natural language processing. As a result, associations with times when there are many positive posts and specific keywords are discovered.

[1305] The server then uses these analyses to extract market trends and patterns in product reviews, such as graphs showing spikes in positive reviews or responses in specific market segments. This information is then sent to users in regular reports, allowing them to make decisions about marketing strategies and product improvements.

[1306] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and is able to quickly provide useful insights to users.

[1307] The processing flow will be explained below.

[1308] Step 1: User-specified data source configuration

[1309] Users tell the system the type of data they want to collect and the data source (e.g., social media platforms or news sites).

[1310] Step 2: Request to start data collection

[1311] The device sends a data collection request to the server based on the user's settings.

[1312] Step 3: Data Acquisition

[1313] The server calls the specified API (e.g., Twitter API, RSS feed) and collects the data. At this time, the URL and query parameters to be accessed are dynamically generated according to the user's settings.

[1314] Example: Collect related posts from the Twitter API based on the user-specified hashtag "market trends."

[1315] Step 4: Data cleansing

[1316] The server cleanses the collected data.

[1317] For text data: Remove HTML tags, unnecessary symbols, and links.

[1318] For image data: unify the resolution and reduce noise.

[1319] For audio data: Apply a noise reduction filter to improve audio clarity.

[1320] Step 5: Analytical processing according to the data format

[1321] The server passes the cleansed data to various AI algorithms for analysis.

[1322] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction from text data.

[1323] Image recognition algorithms are used to recognize specific objects and scenes.

[1324] Using a speech recognition algorithm, the voice data is converted into text and further subjected to sentiment analysis.

[1325] Step 6: Find data correlations

[1326] The server analyzes the relationships between different data sources and extracts important correlations.

[1327] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[1328] Step 7: Creating a visualization

[1329] The server generates dashboards and graphs to visually illustrate the extracted correlations.

[1330] Example: Graphing sentiment analysis results on a timeline.

[1331] Step 8: Generate reports for users

[1332] The server compiles insights from user settings and historical data to create customized reports.

[1333] Example: Generate a weekly report summarizing market trends and sentiment analysis related to a specified hashtag.

[1334] Step 9: Notification and distribution

[1335] The device will notify the user of important announcements and analysis results at specific times.

[1336] Example: Sending reports generated by the server to users via email.

[1337] Example: Analytical results are updated in real time on a dashboard, providing users with a view into the results.

[1338] Example 1

[1339] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1340] Conventional data analysis systems have struggled to provide users with the information they need quickly and accurately, particularly in the complex and time-consuming process of consistently preprocessing data collected from different data sources and extracting useful insights in real time to provide them to users.

[1341] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1342] In this invention, the server includes a data collection means for collecting information from specified data sources using APIs, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data using natural language processing, image recognition, and voice recognition, a knowledge extraction means for identifying correlations between different data sources and specific patterns from the analysis results and extracting important knowledge, and an information provision means for providing the extracted knowledge in a customized format based on user settings and past trends, visualizing it on a dashboard, or sending it to the user as a regular report, thereby enabling the user to quickly and accurately obtain useful insights.

[1343] Below are definitions of key words.

[1344] The "terminal means" is a device for inputting a data source and a request designated by a user and transmitting the input to a server.

[1345] "Data collection means" means a means of obtaining necessary information from a specified data source using an API.

[1346] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary symbols and links, and arranging the data into an appropriate format.

[1347] "Data analysis means" refers to means for analyzing cleansed data using natural language processing, image recognition, and voice recognition.

[1348] "Insight extraction means" refers to means for identifying correlations and specific patterns between different data sources from the results of data analysis and extracting important insights.

[1349] "Information provision means" refers to the means of providing extracted knowledge to users, visualizing it on a dashboard, or sending it as a regular report.

[1350] "API" is an abbreviation for Application Programming Interface, an interface for exchanging data between different software programs.

[1351] "Natural language processing" is a technology that uses computers to understand and process human language.

[1352] "Image recognition" is a technology that identifies specific objects from digital images and videos.

[1353] "Speech recognition" is a technology that converts voice data into text and understands its content.

[1354] A "dashboard" is an interface that provides users with visualized data information.

[1355] A "report" is an information document that summarizes analysis results and findings in written form and provides them to users.

[1356] The present invention provides a system for collecting information from data sources specified by a user, analyzing the data, and providing useful insights. The system includes a terminal unit, a data collection unit, a data preprocessing unit, a data analysis unit, a knowledge extraction unit, and an information provision unit.

[1357] System configuration and operation

[1358] Terminal means

[1359] The user uses the device interface to specify a particular data source (e.g., a social media platform, a news site) and associated hashtags, and this information is sent to the server in the form of a request.

[1360] Data collection methods

[1361] The server uses APIs to gather information from data sources specified by the user, for example, using the Twitter API or a news site's API to gather posts and articles related to a particular hashtag.

[1362] Data preprocessing measures

[1363] The server cleanses the collected data. Specifically, it removes unnecessary symbols and links from text data, standardizes the resolution of image data, and removes noise. It also performs noise reduction and segmentation on audio data. For example, it cleanses text data using the Python pandas library.

[1364] Data Analysis Methods

[1365] The cleansed data is then analyzed by the server. Natural language processing (NLP) is used to perform sentiment analysis on the text data, and image recognition is used to identify specific objects or scenes within images. For audio data, a speech recognition algorithm is used to convert it into text and analyze its content. Specifically, Google's BERT model is used for natural language processing, and OpenCV is used for image recognition.

[1366] Knowledge extraction means

[1367] From the analysis results, the server identifies correlations and specific patterns between different data sources and extracts key insights, such as the correlation between sentiment scores of social media posts and stock prices in market data. This process is powered by the Scikit-learn library.

[1368] Information provision means

[1369] The extracted insights are delivered in a customized format based on user preferences and historical trends, visualized on a dashboard, or sent to users as periodic reports. For example, the dashboards are generated using Tableau, and the reports are generated using Python's ReportLab.

[1370] Specific examples

[1371] If a user wants to analyze the market valuation of a new product, they would follow these steps:

[1372] 1. Users enter the name of a new product and related hashtags into the system and specify Twitter or a news site as the data source.

[1373] 2. The server collects relevant information from these data sources using the specified APIs.

[1374] 3. After the collected data is cleansed, sentiment analysis is performed using natural language processing. For example, Twitter posts are analyzed using the BERT model to classify them as positive, negative, or neutral.

[1375] 4. The server uses these analysis results to extract market trends and patterns of product reviews. It then creates graphs to visualize periods of time when positive reviews spike and reactions in specific market segments. Matplotlib is used to generate these graphs.

[1376] 5. Extracted insights are sent to users in regular reports, with specific insights such as, "Positive posts about new products are most prevalent between 2:00 PM and 4:00 PM."

[1377] Prompt Sentence Examples

[1378] I'd like to analyze the market evaluation of new product XYZ, so I'd like you to collect posts related to XYZ from Twitter and news sites and perform a sentiment analysis. For example, I'd like to know how many positive and negative posts there are.

[1379] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, making it possible to quickly provide useful insights to users.

[1380] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1381] Step 1:

[1382] User request submission

[1383] The user uses the device interface to specify a specific data source (e.g., a social media platform, a news site) and relevant hashtags. Once the user enters this information and presses the send button, the request is sent from the device to the server.

[1384] Input: Data source and hashtags entered by the user into the form (e.g., "Twitter" or "New product name")

[1385] Output: Request data from the terminal to the server

[1386] Specific behavior:

[1387] A user enters the name of a new product and related hashtags into a form.

[1388] The user selects Twitter or a news site.

[1389] The user presses the "Send" button.

[1390] The terminal transmits the request data to the server.

[1391] Step 2:

[1392] Data collection by the server

[1393] Based on the request received, the server uses APIs to gather information from specified data sources, for example, Twitter API or a news site's API to retrieve posts and articles related to a specific hashtag.

[1394] Input: Request data sent from the terminal

[1395] Output: Raw data obtained from each data source

[1396] Specific behavior:

[1397] The server accesses the Twitter API and collects posts containing "new product name."

[1398] The server accesses the news site's API and collects articles related to the keyword "new product name."

[1399] The server stores this raw data internally.

[1400] Step 3:

[1401] Data preprocessing by the server

[1402] The server cleanses the collected data by removing unnecessary symbols and links from text data, standardizing the resolution of image data and removing noise, and performing noise reduction and segmentation on audio data.

[1403] Input: Raw data obtained from each data source

[1404] Output: Cleansed data

[1405] Specific behavior:

[1406] The server removes unnecessary links and symbols from the Twitter posts it retrieves.

[1407] The server extracts the body of the news article and removes any advertisements or sidebar information.

[1408] The server standardizes the resolution of the image data and removes noise.

[1409] Step 4:

[1410] Data analysis by server

[1411] The server analyzes the pre-processed data: natural language processing (NLP) is used to perform sentiment analysis on text data, image recognition is used to identify specific objects on image data, and speech recognition algorithms are used to convert audio data into text and analyze its content.

[1412] Input: Cleansed data

[1413] Output: Analysis result data

[1414] Specific behavior:

[1415] The server performs sentiment analysis on the Twitter posts retrieved by the server using the BERT model, classifying them as positive, negative, or neutral.

[1416] Analyze the titles and text of news articles to extract trending topics and keywords.

[1417] The server performs object recognition on the image data using OpenCV.

[1418] The audio data is converted into text and the content is further analyzed.

[1419] Step 5:

[1420] Knowledge extraction by server

[1421] The server extracts key findings from the data analysis, finding correlations and specific patterns between different data sources and generating useful insights.

[1422] Input: Analysis result data

[1423] Output: Knowledge data

[1424] Specific behavior:

[1425] The server compares the Twitter sentiment analysis results with market stock price data to find correlations between the two.

[1426] The increase or decrease in positive posts during specific time periods is graphed, suggesting applications for marketing.

[1427] Step 6:

[1428] Information provided by the server

[1429] The server provides the extracted insights to the user, visualizing them on a dashboard or sending them to the user as regular reports. Specifically, it generates graphs and tables to visualize the insights and provides the insights in a format that is easy for the user to understand.

[1430] Input: Knowledge data

[1431] Output: Customization information provided to the user

[1432] Specific behavior:

[1433] The server generates a graph of times when there are many positive posts and displays it on the dashboard.

[1434] Generate a weekly report in PDF format and send it to your email address.

[1435] In this way, the system can perform clear steps and provide useful insights to the user quickly.

[1436] (Application example 1)

[1437] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1438] Conventional surveillance systems have difficulty in monitoring a wide area or detecting suspicious individuals or objects in real time. Furthermore, there have been no systems that provide risk information by integrating information collection from social media and sentiment analysis. This has made it difficult to make quick and accurate decisions, and has led to problems in implementing effective security measures.

[1439] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1440] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, a video recognition means for monitoring video data in real time and detecting suspicious objects and people, and a natural language processing means for collecting risk information from social media and performing sentiment analysis. This enables wide-area monitoring and real-time detection of suspicious people and objects, and also enables risk information to be provided by integrating social media information, enabling quick and accurate decision-making.

[1441] A "data collection method" is a method by which a user collects information from a particular data source.

[1442] "Data preprocessing means" refers to means for cleansing collected data, removing unnecessary information, and preparing the data in an appropriate format.

[1443] "Data analysis means" refers to means for analyzing cleansed data to extract specific patterns and trends.

[1444] "Knowledge extraction means" refers to a means for extracting important knowledge from the results of data analysis.

[1445] "Information provision means" refers to a means for providing extracted knowledge to users.

[1446] "Video recognition means" is a means for monitoring video data in real time and detecting suspicious objects and people.

[1447] "Natural language processing means" is a means for collecting risk information from social media and performing sentiment analysis.

[1448] The present invention aims to realize a security service that collects information from data sources specified by a user, analyzes the information, and provides useful insights. The system includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means.

[1449] Data collection methods:

[1450] The server collects real-time video from devices such as surveillance cameras and drones. This video data is used to detect suspicious objects and people. Additionally, posts related to designated hashtags on social media (e.g., Twitter) can be collected via API. This allows users to efficiently collect a wide range of information.

[1451] Data preprocessing methods:

[1452] The server cleanses the collected video and social media data. For video data, it standardizes image quality and removes noise, and for text data, it removes unnecessary symbols and links. This improves the accuracy of the data and increases the efficiency of subsequent analysis.

[1453] Data analysis methods:

[1454] The server applies video recognition algorithms and natural language processing (NLP) to the cleansed data. Video recognition is used to detect suspicious objects and people in real time from surveillance camera footage. NLP is also used to perform sentiment analysis of social media posts and extract potential risk information. These analyses are performed using existing libraries such as OpenCV and TextBlob.

[1455] Knowledge extraction means:

[1456] The server extracts important insights from the analysis results. For example, it integrates location information of suspicious objects detected from video data and risk information extracted from social media, and finds correlations. This allows users to make accurate decisions in real time.

[1457] Information provision method:

[1458] The server uses a head-mounted display (HMD) to provide extracted insights to users, showing the location of suspicious objects and people in real time and providing social media sentiment analysis results as warnings, allowing users to take prompt and appropriate action.

[1459] Examples:

[1460] For example, if a security camera monitors an area in real time and detects a suspicious individual, this information will be sent to security guards via the HMD. Meanwhile, if social media posts related to the same area are analyzed and any disturbing information is found, this information will also be sent to security guards. This allows security guards to grasp the overall situation and respond quickly.

[1461] Example prompt sentence:

[1462] Below is an example of a prompt for a system using the present invention:

[1463] "I want to create an AI model that collects video data from surveillance cameras in real time and detects suspicious people and objects. I want to preprocess the collected video data and analyze it using the AI ​​model. How can I then build a system that displays the location information and warnings of detected suspicious objects in real time?"

[1464] "I want to collect posts related to a specified hashtag on Twitter and perform sentiment analysis. How can I build a system that extracts risk information in a specific area and issues real-time alerts to security guards?"

[1465] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1466] Step 1:

[1467] Data collection

[1468] The server acquires real-time video data from devices such as surveillance cameras and drones. The input is a video data source such as a surveillance camera, and the server captures and saves it frame by frame. In parallel, it uses the specified social media API to collect posts containing related hashtags and keywords. In this case, the input is the hashtag or keyword, and the output is the related SNS post data.

[1469] Step 2:

[1470] Data Preprocessing

[1471] The server cleanses the collected video data. Specific operations include unifying the resolution of the video data, removing noise, and improving image quality. The input is the video data collected in step 1, and the output is the preprocessed video data. For social media data, unnecessary symbols and links are deleted and text data is cleaned. The input is text data collected from SNS, and the output is preprocessed text data.

[1472] Step 3:

[1473] Data analysis

[1474] The server applies a video recognition algorithm to the preprocessed video data. Specifically, it performs object detection to detect suspicious objects and people for each frame. The input is the preprocessed video data, and the output is information about the detected suspicious objects and people. In addition, it performs sentiment analysis on the preprocessed text data using natural language processing (NLP). This allows it to obtain positive or negative sentiment scores as output from the processed text data as input.

[1475] Step 4:

[1476] Knowledge extraction

[1477] The server extracts important insights from the results of the data analysis. Specifically, it extracts location information of suspicious objects detected from video data and behavioral patterns of suspicious individuals. The input is the analysis results obtained in step 3, and the output is the extracted insights (detailed information on suspicious objects and individuals). Similarly, it extracts posts with high negative scores in specific regions from the results of social media sentiment analysis.

[1478] Step 5:

[1479] Providing information

[1480] The server uses a head-mounted display (HMD) to provide the extracted insights to the user. Specifically, it displays the location information of suspicious objects and people on the HMD in real time and issues warnings based on the results of social media sentiment analysis. The input is the insights extracted in step 4, and the output is the information and warnings displayed on the HMD. This allows the user to take prompt and appropriate action.

[1481] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1482] The present invention is a system that includes a data collection means, a data preprocessing means, a data analysis means, a knowledge extraction means, and an information provision means, as well as an emotion engine that recognizes user emotions. This system has a mechanism for collecting information from data sources specified by the user, analyzing that data, and providing useful insights. It also has a function for customizing the information provided by taking the user's emotions into consideration.

[1483] Data collection methods:

[1484] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[1485] Data preprocessing methods:

[1486] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[1487] Data analysis methods:

[1488] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using a speech recognition algorithm, and its content is then further analyzed.

[1489] Knowledge extraction means:

[1490] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[1491] Information provision method:

[1492] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the information is customized based on the user's sentiment, and visualized on a dashboard or sent to the user as regular reports. Users can then make quick and accurate decisions based on this information.

[1493] Emotion Engine:

[1494] It has the ability to recognize the user's emotions in real time and adjust the information it provides based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that will reduce stress.

[1495] Specific examples

[1496] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies Twitter and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[1497] For example, if the emotion engine detects that a user is feeling anxious, it will create a report centered around positive reviews and success stories, providing information that will reduce the user's stress, allowing the user to make decisions about marketing strategies and product improvements with peace of mind.

[1498] In this way, the system of the present invention automates the entire process from data collection to providing knowledge, and further customizes information taking into account the user's emotions, making it possible to quickly provide the most useful insights to the user.

[1499] The processing flow will be explained below.

[1500] Step 1: Specify the data source

[1501] Users tell the system the type of data they want to collect (e.g., text, images, audio) and the corresponding data source (e.g., social media platform, news site).

[1502] Step 2: Submit a request for data collection

[1503] The device sends a data collection request to the server based on the user's specifications, including the data source URL and query parameters.

[1504] Step 3: Collect data

[1505] The server calls the specified API (e.g., Twitter API, RSS feed) to collect data, which is then temporarily stored.

[1506] Step 4: Cleanse the data

[1507] The server cleanses the collected data.

[1508] For text data: Remove HTML tags, unnecessary symbols, and links and purify the data into text.

[1509] For image data: The resolution is unified and noise is removed.

[1510] For audio data: Apply a noise reduction filter to improve audio clarity.

[1511] Step 5: Data analysis

[1512] The server passes the cleansed data to various AI algorithms for analysis.

[1513] Natural language processing (NLP) is used to perform sentiment analysis and topic extraction of text data.

[1514] Image recognition algorithms identify specific objects and scenes.

[1515] A speech recognition algorithm converts the voice data into text and then analyzes the content.

[1516] Step 6: Extracting insights

[1517] The server extracts key insights from the results of data analysis, which includes finding correlations between different data sources and pattern recognition.

[1518] Example: Analyzing the correlation between sentiment scores of social media posts and stock prices from market data.

[1519] Step 7: Recognizing user emotions

[1520] The emotion engine analyzes the user's real-time emotional state, including emotion analysis from voice data and emotion inference from user input behavior.

[1521] Step 8: Customize your information offering

[1522] The server adjusts the information provided to the user based on the emotional state obtained by the emotion engine.

[1523] Example: If a user is in an anxious state, prioritize positive information.

[1524] Step 9: Provide information

[1525] The server provides tailored insights to users, including visualization in a dashboard and the generation and delivery of periodic reports.

[1526] Information is updated in real time on the dashboard and displayed visually in an easy-to-understand manner.

[1527] The analysis results will be emailed to the user as a weekly report.

[1528] Step 10: User feedback

[1529] Users make decisions based on the information provided and input their results and feedback into the system, which allows the system to learn and reflect this in the next information provided.

[1530] The above is the specific processing flow in the system of the present invention. In this way, it becomes possible to provide useful insights to users quickly and accurately.

[1531] Example 2

[1532] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1533] Conventional data analysis systems process and provide collected data in a uniform manner, which means they are unable to consider the emotional state or individual needs of users and are unable to support optimal decision-making.In addition, the wide variety of data sources makes it difficult to efficiently cleanse and analyze collected data.

[1534] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1535] In this invention, the server includes a data collection means, a preprocessing means for cleansing the collected data, an analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing the user's emotions and adjusting information based on that state. This makes it possible to provide information customized according to the user's emotional state, thereby supporting faster and more effective decision-making.

[1536] "Data Collection Implement" means any device or software used to collect information from user-specified data sources.

[1537] A "pre-processing means" is a device or software that cleanses the collected data and prepares it in a form suitable for analysis.

[1538] "Analysis means" refers to a device or software that analyzes the cleansed data using various algorithms to extract useful information and patterns.

[1539] The "knowledge extraction means" is a device or software for extracting important insights and meaningful knowledge from the analysis results obtained by the analysis means.

[1540] "Information provision means" refers to devices or software that provide extracted knowledge to users in an easy-to-understand manner. Specifically, this includes visualization on a dashboard and generation of reports.

[1541] An "emotion engine" is a device or software that recognizes a user's emotional state in real time and adjusts the information provided based on that state.

[1542] "API" stands for Application Programming Interface, a standardized way of sending and receiving data between different software applications.

[1543] "Natural language processing" is the technology for understanding and analyzing human language, and is used for sentiment analysis and semantic analysis of text.

[1544] "Image recognition" is the technology of identifying specific objects or patterns from image data.

[1545] "Speech recognition" is a technology that converts voice data into text and analyzes its content.

[1546] The present invention is a system that includes a data collection means, a preprocessing means, an analysis means, a knowledge extraction means, an information provision means, and an emotion engine. This allows the system to collect information from a data source specified by a user, analyze the data, and provide useful knowledge. The system also has a function to adjust the information taking the user's emotions into account.

[1547] Data collection methods

[1548] When a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the required information from the specified data source. For example, if a user specifies a specific hashtag, the server retrieves posts related to that hashtag. This collected data is then stored in the server's internal database.

[1549] Pretreatment means

[1550] The collected data is cleansed by the server. In the case of text data, unnecessary symbols and links are removed and the data is formatted appropriately. In the case of image data, the resolution is standardized and noise is removed. In the case of audio data, noise reduction and segmentation are performed. This preprocessing converts the data into a format suitable for analysis.

[1551] analytical means

[1552] The cleansed data is then analyzed by the server using various AI algorithms. Specifically, natural language processing (NLP) is used to perform sentiment analysis on text data, and image recognition is used to identify specific objects or scenes in images. Voice data is converted into text using speech recognition algorithms, and its content is analyzed. These algorithms are implemented using generative AI models.

[1553] Knowledge extraction means

[1554] The server extracts key insights from the analysis results, for example, by finding correlations between different data sources and analyzing the correlation between sentiment scores of social media posts and stock prices from market data, allowing users to gain deeper insights.

[1555] Information provision means

[1556] The extracted insights are delivered in a customized format based on the user's settings and past trends. The server visualizes these insights on a dashboard or sends them to the user as regular reports, allowing the user to make quick and accurate decisions based on the information provided.

[1557] Emotion Engine

[1558] The emotion engine recognizes the user's emotions in real time and adjusts the information provided based on that state. For example, if the user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress. This allows the system to provide the most appropriate information for the user.

[1559] Specific examples

[1560] Suppose a user wants to analyze the market evaluation of a new product. In this case, the user enters the name of the new product and related hashtags into the system and specifies social media platforms and news sites as data sources. The server collects relevant information from these data sources using the specified API. The collected data is cleansed and then subjected to sentiment analysis using natural language processing. Furthermore, the server recognizes the user's emotional state using an emotion engine and provides information tailored to the user's mood.

[1561] Prompt Sentence Examples

[1562] For example, a user might prompt a generative AI model as follows:

[1563] "I want to analyze the market evaluation of a new product using social media platforms and news sites as data sources. The related hashtag is new product 2023. Based on the results of the sentiment engine, please create a report with positive evaluations and success stories."

[1564] In this way, by automating the entire process from data collection to providing insights, and further customizing information taking into account user emotions, we have created a system that can quickly provide the most useful insights to users.

[1565] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1566] Step 1:

[1567] The user specifies a particular data source.

[1568] Users enter the required keywords or hashtags into the system's interface.

[1569] For example, a user enters the hashtag "New Products 2023."

[1570] Input data: hashtag "New Products 2023"

[1571] Output: API request data as a user request

[1572] Specific operation: When the user clicks the search button, the device converts the entered data into an API request format and sends it to the server.

[1573] Step 2:

[1574] The terminal sends the user's request to the server.

[1575] The device converts the request, including the hashtags and keywords entered by the user, into JSON format.

[1576] Input data: User-entered keywords (API request data)

[1577] Output: API request sent to the server

[1578] Specific operation: The device sends an API request to the server in the form of an HTTP request.

[1579] Step 3:

[1580] The server uses the specified API to collect information from the data source.

[1581] The server sends a request to the API endpoint of the social media platform or news site to retrieve the required data.

[1582] Input data: API request

[1583] Output: Data collected from social media platforms and news sites (text, images, audio, etc.)

[1584] Specific operation: The server sends a request to the API endpoint and stores the returned JSON data in an internal database.

[1585] Step 4:

[1586] The server cleanses the collected data.

[1587] The server removes unnecessary symbols and links from the text data and formats it appropriately. For image data, it standardizes the resolution and removes noise. For audio data, it performs noise reduction and segmentation.

[1588] Input data: raw data collected

[1589] Output: Cleansed data (clean text, images, audio data)

[1590] Specific operation: The server executes the data cleansing algorithm, filtering and format conversion.

[1591] Step 5:

[1592] The server analyzes the cleansed data.

[1593] The server uses natural language processing (NLP) to perform sentiment analysis on the text data, image recognition to identify specific objects and scenes in images, and speech recognition algorithms to convert audio data into text and analyze it.

[1594] Input data: Cleansed data (text, image, audio data)

[1595] Output: Analysis results (emotion scores, recognition results, text conversion data, etc.)

[1596] Specific operation: The server runs NLP algorithms and image recognition algorithms to generate emotion scores and object identification results.

[1597] Step 6:

[1598] The server extracts key insights from the analysis results.

[1599] The server analyzes correlations between different data sources and extracts key insights and findings.

[1600] Input data: Analysis results

[1601] Output: Extracted findings (correlations, patterns, insights, etc.)

[1602] What it does: The server analyzes the analysis results in the database and runs algorithms to extract key insights.

[1603] Step 7:

[1604] The server customizes the insights to provide to the user.

[1605] The server customizes the extracted insights based on the user's past preferences and tendencies.

[1606] Input data: extracted knowledge, user setting data

[1607] Output: Customized insights (reports and dashboards tailored to the user)

[1608] Specific operation: The server retrieves the user's profile data and uses the extracted insights to customize it.

[1609] Step 8:

[1610] The server uses an emotion engine to recognize the user's emotions in real time.

[1611] The emotion engine analyzes the user's emotional state from their facial expressions and voice.

[1612] Input data: Real-time facial expression data and voice data of the user

[1613] Output: User's emotional state (positive, negative, etc.)

[1614] Specific operation: The emotion engine runs facial expression recognition algorithms and voice analysis algorithms to evaluate the user's emotions.

[1615] Step 9:

[1616] The server tailors the information based on the user's emotional state.

[1617] The server adjusts the content of the information it provides depending on the recognized emotional state.

[1618] Input data: user's emotional state, customized insights

[1619] Output: Emotionally adjusted information (reports and feedback containing positive information)

[1620] Specific operation: The server filters information based on the emotional state, selects appropriate information and provides it to the user.

[1621] Step 10:

[1622] The server provides the information to the user.

[1623] The server visualizes the information on a dashboard or sends it to the user as periodic reports.

[1624] Input data: emotion-modulated information

[1625] Output: The final information provided to the user (dashboard display, report sending, etc.)

[1626] What it does: The server generates graphs and tables to visualize the information and displays them on the user's dashboard. It also generates a report in PDF format and emails it to the user.

[1627] This will automate the entire process from data collection to providing insights, and by customizing information taking into account user emotions, a system will be created that supports quick and accurate decision-making.

[1628] (Application example 2)

[1629] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1630] Conventional data collection and analysis systems are unable to provide information that takes into account the user's emotions, which can result in users receiving inappropriate information. Furthermore, there is no system in place to recognize customer emotions in real time and respond appropriately based on those emotions, making it difficult for store staff to respond quickly and accurately to customer needs.

[1631] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1632] In this invention, the server includes a data collection means, a data preprocessing means for cleansing the collected data, a data analysis means for analyzing the cleansed data, a knowledge extraction means for extracting important knowledge from the analysis results, an information provision means for providing the extracted knowledge to the user, and an emotion engine for recognizing customer emotions in real time and customizing the information to be provided. This makes it possible to provide the most appropriate information to the user and to present an appropriate response method according to the customer's emotions.

[1633] 1. "Data Collection Implement" means a device or system capable of collecting information from user-specified data sources.

[1634] 2. "Data preprocessing means" refers to a device or system that removes unnecessary information and cleanses data in order to prepare collected data in an appropriate format.

[1635] 3. "Data Analysis Tool" means a device or system that uses AI algorithms to analyze cleansed data and extract useful insights.

[1636] 4. A "Insight Extraction Tool" is a device or system used to find significant patterns and correlations from the results of data analysis.

[1637] 5. "Information Delivery Measure" means a device or system that delivers extracted insights to a user in a customized format based on the user's preferences and past trends.

[1638] 6. An "emotion engine" is a device or system that has the ability to recognize the emotions of users or customers in real time and adjust the information it provides based on that state.

[1639] 7. "API" means an interface for exchanging functions and data between different software programs, and is used to collect information from specified data sources.

[1640] 8. "Natural language processing" is a technology for analyzing text data and performing sentiment analysis and understanding intent.

[1641] 9. “Image recognition” is a technology for identifying specific objects or scenes from image data.

[1642] 10. "Speech recognition" is a technology for converting voice data into text and analyzing its content.

[1643] The system of the present invention mainly includes the following means: data collection means, data preprocessing means, data analysis means, knowledge extraction means, information provision means, and an emotion engine. This system has a mechanism for collecting information from data sources specified by the user, analyzing the data, and providing useful insights. It also has a function for recognizing the emotions of users and customers in real time and customizing the information provided based on those emotions.

[1644] First, when a user specifies a specific data source (e.g., a social media platform or news site), the device sends the request to the server. The server uses an API to retrieve the necessary information from the specified data source. For example, if a user specifies the use of a specific hashtag, the server retrieves posts related to that hashtag.

[1645] The collected data is cleansed by the server. For text data, unnecessary symbols and links are removed and the data is formatted appropriately. For image data, the resolution is standardized and noise is removed. For audio data, noise reduction and segmentation are performed.

[1646] The cleansed data is then analyzed by the server using various AI algorithms, such as natural language processing (NLP) to analyze the sentiment of text data and image recognition to identify specific objects or scenes within images. Voice data is converted into text using speech recognition algorithms, and its content is then further analyzed.

[1647] The server extracts key insights from the data analysis, which can include finding correlations between different data sources and identifying specific patterns, such as the correlation between sentiment scores of social media posts and stock prices from market data.

[1648] The extracted insights are delivered in a customized format based on the user's preferences and past trends. Furthermore, the emotion engine recognizes the user's and customer's emotions in real time and adjusts the information provided based on their state. For example, if a user is in an unstable emotional state, it will prioritize positive information and provide feedback that reduces stress.

[1649] A specific example is when store staff use their smartphones to capture customers' facial expressions and tone of voice while serving them, and then suggest appropriate ways to respond based on the analysis results. This allows staff to suggest products and provide services that are appropriate to the customer's emotions.

[1650] Example prompt sentence:

[1651] "Identify the specific emotion this customer is experiencing and, based on that, suggest the best product or customer service approach."

[1652] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1653] Step 1:

[1654] The user specifies a specific data source (e.g., a social media platform or a news site) and sends a request to the device.

[1655] Input: User-specified data source and associated parameters (e.g., hashtags)

[1656] Output: Request data from the terminal to the server

[1657] How it works: The user interacts with the application and inputs the data source to be investigated and related parameters. The device then sends this information to the server.

[1658] Step 2:

[1659] The server uses the API to collect information from the specified data sources.

[1660] Input: Request data sent from the terminal

[1661] Output: Raw data retrieved from the specified data source.

[1662] What it does: The server makes an API request to gather information from the specified data source (e.g., Twitter API), including related posts and articles.

[1663] Step 3:

[1664] The server cleanses the collected data.

[1665] Input: Raw data collected

[1666] Output: Cleansed data

[1667] How it works: The server removes unnecessary symbols and links from text data, standardizes the resolution of image data, and reduces noise in audio data, thereby standardizing the data.

[1668] Step 4:

[1669] The server analyzes the cleansed data.

[1670] Input: Cleansed data

[1671] Output: Analysis results (e.g., sentiment scores, identified objects)

[1672] How it works: The server uses natural language processing (NLP) algorithms to perform sentiment analysis on text data, image data to analyze using image recognition algorithms, and audio data to convert it into text using speech recognition algorithms, which then analyzes its content.

[1673] Step 5:

[1674] The server extracts important insights from the analysis results.

[1675] Input: Analysis results

[1676] Output: Key findings (e.g. correlations, specific patterns)

[1677] How it works: The server finds correlations between different data sources and identifies specific patterns, such as the correlation between sentiment scores from social media posts and stock prices from market data.

[1678] Step 6:

[1679] The server provides the extracted knowledge to the user.

[1680] Input: Key Findings

[1681] Output: Customized information (e.g. reports, dashboards)

[1682] How it works: The server customizes insights based on user preferences and historical trends, visualizes them on a dashboard, or sends them to the user as regular reports.

[1683] Step 7:

[1684] The server uses an emotion engine to recognize the user's emotions in real time and adjusts information based on that state.

[1685] Input: User emotion data (e.g., facial expressions, tone of voice)

[1686] Output: Tailored information

[1687] How it works: The server uses a camera and microphone to capture and analyze the customer's facial expressions and tone of voice. Based on the analysis results, it provides information that corresponds to the user's emotional state. For example, if the user is anxious, it prioritizes positive information.

[1688] Step 8:

[1689] Users and store staff receive the information provided by the server and apply it to their actual work.

[1690] Input: Customized information provided by the server

[1691] Output: Real-world decisions and actions

[1692] How it works: Users and store staff decide business policies based on reports and dashboards provided by the server, and reflect these in customer service and marketing strategies.

[1693] This allows users to receive relevant insights in a timely manner, enabling them to provide better services and make better decisions based on customer sentiment.

[1694] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1695] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1696] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1697] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1698] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1699] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1700] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1701] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1702] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1703] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1704] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1705] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1706] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1707] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1708] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1709] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1710] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1711] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1712] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1713] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1714] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1715] The following is further disclosed regarding the above embodiment.

[1716] (Claim 1)

[1717] data collection means;

[1718] a data preprocessing means for cleansing the collected data;

[1719] a data analysis means for analyzing the cleansed data;

[1720] knowledge extraction means for extracting important knowledge from the analysis results;

[1721] an information providing means for providing extracted knowledge to users;

[1722] A system including:

[1723] (Claim 2)

[1724] 10. The system of claim 1, wherein the system collects information from specified data sources using an API.

[1725] (Claim 3)

[1726] 10. The system of claim 1, wherein the cleansed data is analyzed using natural language processing, image recognition, and speech recognition.

[1727] "Example 1"

[1728] (Claim 1)

[1729] terminal means for specifying a data source based on a user request and transmitting the request to a server;

[1730] a data collection means for collecting information from specified data sources using an API;

[1731] a data preprocessing means for cleansing the collected data;

[1732] a data analysis means for analyzing the cleansed data using natural language processing, image recognition, and voice recognition;

[1733] A knowledge extraction method that identifies correlations and specific patterns between different data sources from the analysis results and extracts important knowledge;

[1734] An information delivery method that delivers extracted insights in a customized format based on user settings and past trends, visualizes them on a dashboard, or sends them to users as regular reports;

[1735] A system including:

[1736] (Claim 2)

[1737] 10. The system of claim 1, wherein the system collects information from specified data sources using an API.

[1738] (Claim 3)

[1739] 10. The system of claim 1, wherein the system analyzes the cleansed data using natural language processing, image recognition, and speech recognition to identify correlations and specific patterns between different data sources.

[1740] "Application Example 1"

[1741] (Claim 1)

[1742] data collection means;

[1743] a data preprocessing means for cleansing the collected data;

[1744] a data analysis means for analyzing the cleansed data;

[1745] knowledge extraction means for extracting important knowledge from the analysis results;

[1746] an information providing means for providing extracted knowledge to users;

[1747] A video recognition means for monitoring video data in real time and detecting suspicious objects and people;

[1748] A natural language processing means for collecting risk information from social media and performing sentiment analysis;

[1749] A system including:

[1750] (Claim 2)

[1751] 10. The system of claim 1, wherein the system collects information from specified data sources using an API.

[1752] (Claim 3)

[1753] 10. The system of claim 1, wherein the cleansed data is analyzed using natural language processing, image recognition, and speech recognition.

[1754] "Example 2: Combining Emotion Engines"

[1755] (Claim 1)

[1756] data collection means;

[1757] a pre-processing means for cleansing the collected data;

[1758] an analytical means for analyzing the cleansed data;

[1759] knowledge extraction means for extracting important knowledge from the analysis results;

[1760] an information providing means for providing extracted knowledge to users;

[1761] An emotion engine that recognizes the user's emotions and adjusts information based on their state;

[1762] A system including:

[1763] (Claim 2)

[1764] 10. The system of claim 1, wherein the system collects information from specified data sources using an API.

[1765] (Claim 3)

[1766] 10. The system of claim 1, wherein the cleansed data is analyzed using natural language processing, image recognition, and speech recognition.

[1767] "Application example 2 when combining emotion engines"

[1768] (Claim 1)

[1769] data collection means;

[1770] a data preprocessing means for cleansing the collected data;

[1771] a data analysis means for analyzing the cleansed data;

[1772] knowledge extraction means for extracting important knowledge from the analysis results;

[1773] an information providing means for providing extracted knowledge to users;

[1774] An emotion engine that recognizes customer emotions in real time and customizes the information provided;

[1775] A system including:

[1776] (Claim 2)

[1777] 10. The system of claim 1, wherein the system collects information from specified data sources using an API.

[1778] (Claim 3)

[1779] 10. The system of claim 1, wherein the cleansed data is analyzed using natural language processing, image recognition, and speech recognition.

[1780] (Claim 4)

[1781] 10. The system of claim 1, which captures a customer's facial expressions and tone of voice and suggests an appropriate response method based on the analysis results. [Explanation of symbols]

[1782] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. data collection means; a data preprocessing means for cleansing the collected data; a data analysis means for analyzing the cleansed data; knowledge extraction means for extracting important knowledge from the analysis results; an information providing means for providing extracted knowledge to users; A system including:

2. The system of claim 1 , wherein the system uses an API to collect information from specified data sources.

3. The system of claim 1 , wherein the cleansed data is analyzed using natural language processing, image recognition, and speech recognition.

Images