system

Abstract

We provide the system. [Solution] A means for receiving structured and unstructured data and storing them in data storage, An analysis method for converting unstructured data into structured data, Processing methods for performing data analysis based on structured data, A prediction method that performs predictions based on the analysis results, Control measures for formulating and executing forecast-based strategies, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional enterprise-oriented data management systems mainly rely only on structured data, and it is difficult to effectively utilize unstructured data. As a result, there is a problem that business information accumulated in various forms cannot be comprehensively analyzed and utilized for future strategic planning.

Means for Solving the Problems

[0005] This invention provides a means for centrally managing structured and unstructured data. This system enables complex analysis by analyzing unstructured data and converting it into structured data. Furthermore, it includes means for performing data analysis based on structured data and using the results to make future predictions, as well as control means for formulating and executing strategies based on those predictions. In this way, the aim is to enable companies to comprehensively and efficiently utilize data and enhance their decision-making processes.

[0006] "Structured data" refers to data that is stored in an orderly manner according to a predefined data model, and is in a format that can be easily organized into database tables, spreadsheets, and other similar formats.

[0007] "Unstructured data" refers to data that is stored without adhering to a predefined structure, and includes formats such as text, audio, images, and video.

[0008] "Data storage" refers to memory devices and systems used to store data, and possesses the functions of storing, retrieving, and managing data.

[0009] "Analysis means" refers to methods and functions for analyzing the structure and characteristics of data in order to process the data and obtain useful information.

[0010] "Processing means" refers to the part of a computer that has the function of manipulating data and performing specific processing, and it refers to the process from data input to output.

[0011] "Predictive means" refers to methods or functions for estimating future outcomes or trends based on past and present data.

[0012] "Control means" refers to functions or mechanisms that manage various functions and processes of a system and instruct them to operate in accordance with a specific purpose.

[0013] "Communication methods" refer to interfaces and functions that enable systems to exchange information, and include wired and wireless communication methods. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention relates to a system for integrating and managing structured and unstructured data and executing truly useful business strategies through data analysis and forecasting. The system includes processes for data ingestion, analysis of unstructured data, data analysis, forecasting, and strategy execution.

[0036] The server is designed to receive both structured and unstructured data and stores this data in data storage. Unstructured data is converted into structured data using specific analytical methods. Through this analysis, the unstructured data is formatted into a format that is easily visualized and measured using techniques such as natural language processing and image recognition.

[0037] Next, the server performs data analysis using the analyzed structured data. This is a process to extract useful information and trends from past data. The insights gained through this analysis process are used to estimate future outcomes using predictive tools. For example, this can be used to forecast sales or customer behavior.

[0038] Users can formulate specific business strategies based on analysis results and predictions provided by the server. They can then send the selected strategy to the system via their terminal to initiate its execution. The server receives the command to execute the strategy and implements it effectively through its control mechanisms.

[0039] As a concrete example, consider a scenario where a user uploads unstructured audio data of customer feedback to the system. The server converts this audio data into text and uses natural language processing to analyze the customer's emotions and intentions. Based on these results, the server predicts increases or decreases in sales and the need for product improvements. The user then develops a new marketing campaign based on these predictions and sends execution commands through the system. The server manages the resources necessary to run the campaign and deploys it at the optimal time.

[0040] Thus, the system of the present invention enables companies to comprehensively utilize data and make quick and effective decisions.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] Users upload structured data (e.g., database export files) and unstructured data (e.g., audio files and text documents) to the server. The server receives these files and stores the structured data in the `structured_data` list and the unstructured data in the `unstructured_data` list.

[0044] Step 2:

[0045] The server initiates a process to analyze unstructured data and convert it into structured data. For example, after converting audio data to text, it uses natural language processing to extract keywords and perform sentiment analysis from the text. The server then adds the results of this conversion to the structured_data list as structured information.

[0046] Step 3:

[0047] The server performs data analysis based on structured data lists. This analysis includes applying statistical analysis and machine learning algorithms to gain useful insights from the data. For example, it identifies best-selling products and seasonal trends based on historical sales data.

[0048] Step 4:

[0049] The server makes future predictions based on the results of data analysis. This includes using predictive models to forecast future sales and estimate market trends. The prediction results are provided to users to help them in strategic planning.

[0050] Step 5:

[0051] The user develops a specific business strategy based on the prediction results from the server. The strategy is then sent to the server as a command using a terminal, initiating the strategy's execution process.

[0052] Step 6:

[0053] The server executes the plan based on the strategic directives it receives. This execution includes managing the optimal deployment of campaigns and promotions while coordinating with external systems. The server monitors overall progress and makes adjustments to ensure success.

[0054] (Example 1)

[0055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0056] Traditional data management systems have faced challenges in effectively integrating structured and unstructured information and quickly reflecting it in actual business strategies. Unstructured information is diverse in format and difficult to analyze, resulting in significant time and effort required to gain practical insights. Furthermore, there has been a lack of efficient means to execute strategies based on future predictions.

[0057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0058] In this invention, the server includes means for receiving structured and unstructured information and storing it in information memory; means for analyzing unstructured information to convert it into structured information; means for processing information based on the structured information; means for analyzing unstructured information using natural language processing or image recognition technology; and means for transmitting policy execution commands through a terminal. This enables rapid decision-making and effective implementation of strategies by comprehensively utilizing structured and unstructured information.

[0059] "Structured information" refers to information organized according to a specific format or structure, and is typically stored in databases, spreadsheets, and other similar formats.

[0060] "Unstructured information" refers to information that does not adhere to a specific format and exists in a variety of forms, including audio data, image data, and text documents.

[0061] "Information storage" refers to physical or digital storage devices for holding structured and unstructured information.

[0062] "Information analysis means" refers to technologies and tools that have the function of analyzing unstructured information and converting it into structured information.

[0063] "Processing means for performing information analysis" refers to technologies that have the function of performing calculations and models to derive trends and patterns based on structured information.

[0064] "Natural language processing" refers to the technology that enables computers to understand, interpret, and generate human language.

[0065] "Image recognition technology" refers to the technology used to analyze image data and recognize specific objects or features.

[0066] A "terminal" refers to a digital device used by a user to communicate with a server and perform operations or transmit information.

[0067] A "policy execution command" refers to an instruction that prompts a server to begin carrying out specific actions or procedures.

[0068] This invention is a system that enables companies to integrate and utilize structured and unstructured information to formulate and execute useful business strategies. Specific embodiments are described below.

[0069] The server performs its primary functions, primarily information management. First, the server receives structured and unstructured information from users and stores it in its information storage device. Structured information is retrieved from existing databases, while unstructured information includes various data formats such as audio, images, and text. This unstructured information is analyzed by the server using natural language processing and image recognition technologies and converted into structured information. This process utilizes application-specific analysis software.

[0070] The analyzed information is processed on an information analysis platform within the server. This platform uses advanced machine learning algorithms, including generative AI models, to extract customer behavior patterns and market trends from past data. Based on these results, the server predicts future outcomes and provides them to the user.

[0071] Users receive analysis results and predictions from the server via their terminals and formulate new business strategies based on them. In this process, users send instructions from their terminals to the server to prompt specific actions.

[0072] As a concrete example, when a user uploads audio data of customer feedback to a server, the server converts the data into text and performs a sentiment analysis of the customer through natural language processing. Based on the insights gained from this analysis, the user can decide on strategies for increasing sales or improving products. They then send these strategies to the server as execution commands, and the server carries out campaigns and operations according to those commands.

[0073] An example of a prompt for a generative AI model is, "Explain the process of converting audio data into text and using that content to make sales forecasts." This system supports companies in making quick and effective decisions by utilizing information from multiple perspectives.

[0074] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0075] Step 1:

[0076] Users upload structured and unstructured information to the server using their devices. Unstructured information includes customer voice data and email text. This information is stored in a data storage device by the server. Inputs are in various data formats, and these data are securely recorded as outputs.

[0077] Step 2:

[0078] The server uses natural language processing and image recognition technologies to analyze unstructured information. Specifically, it converts audio data into text and uses natural language processing tools to analyze customer emotions and intentions. The input is uploaded unstructured information, and the output is analyzed structured information.

[0079] Step 3:

[0080] The server performs information analysis based on structured information. This includes processes that utilize machine learning algorithms, such as generative AI models, to derive useful trends and patterns from past data. The input is structured information, and the output is the analyzed insights and patterns.

[0081] Step 4:

[0082] The server uses prediction tools based on the analysis results to estimate future outcomes. It performs simulations of sales trends and customer behavior, quantifying future prospects. The input is the analyzed information, and the output is the predicted future results.

[0083] Step 5:

[0084] Users receive prediction results from the server and formulate new business strategies based on the information displayed on their devices. These strategies are then reflected in specific policies, such as the planning of more proactive marketing campaigns. The input is the prediction results, and the output is the formulated strategy.

[0085] Step 6:

[0086] The user sends the formulated strategy to the server as an execution command via their terminal. The server receives this command and begins implementing the policy using its control mechanisms. The input is the command from the user, and the output is the specific actions taken to implement it.

[0087] (Application Example 1)

[0088] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0089] In today's business environment, it is crucial to extract useful information from diverse data and formulate and execute concrete strategies. However, in situations where structured and unstructured data coexist, efficiently analyzing them and providing customized proposals for individual customers is difficult. Therefore, companies are required to utilize data quickly and effectively to enhance their competitiveness.

[0090] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0091] In this invention, the server includes means for receiving structured and unstructured data and storing it in a data storage device, analysis means for converting unstructured data into structured data, and processing means for performing data analysis based on the structured data. This makes it possible to propose products optimized for the customer.

[0092] "Structured data" refers to data that exists in a well-organized, structured format, such as in databases or spreadsheets, making it easy to search and analyze.

[0093] "Unstructured data" refers to data in formats that do not have a specific structure, such as audio, text, and image data, and is difficult to analyze in its original form.

[0094] A "data storage device" is a physical or virtual storage device used to store received data.

[0095] "Analysis methods" refer to the process of processing data using technologies such as natural language processing and image recognition in order to convert unstructured data into structured data.

[0096] "Processing means" refers to a function that uses structured data to perform statistical analysis and machine learning-based data analysis to extract useful information.

[0097] A "predictive function" is a technology that estimates future trends and outcomes based on analyzed data and generates prediction results.

[0098] The "control function" is the process of issuing commands to execute strategies formulated based on prediction results and managing their progress.

[0099] The "suggestion function" is a system that provides optimized product information and purchase suggestions based on customer history and feedback.

[0100] A "communication function" is an interface for exchanging data with an external information processing device.

[0101] The system of the present invention integrates the functions of data reception, analysis, interpretation, prediction, proposal, and execution. The server receives structured and unstructured data and stores it in a data storage device. Unstructured data is converted into structured data using natural language processing and image recognition technologies, and this is carried out through analysis means. The technologies used can be combined with conventional database management systems and include NLP engines and image recognition software.

[0102] The server performs data analysis using the transformed structured data and makes future predictions using machine learning models based on the features extracted from the data. Machine learning frameworks such as TENSORFLOW® and PyTorch are utilized for this purpose. For example, it can recommend the most suitable products and services for the future based on customer purchase history and feedback.

[0103] The terminal operates a generating AI model based on user instructions, formulates a strategy based on the prediction results, and sends commands to the server for execution. An example of a prompt might be: "Analyze past customer feedback and generate a list of products they are most likely to purchase next. Pay particular attention to feedback regarding sizing and product color from unstructured voice data."

[0104] As a result, this information can be used to develop new marketing campaigns and optimize pricing, enabling companies to respond flexibly and quickly to customer needs.

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The server receives customer feedback (unstructured data) in the form of voice and text, as well as purchase history (structured data), from smartphone devices. The received data is stored in a data storage device. Here, the input is voice data and past purchase history, and the output is the raw data stored in the data storage device.

[0108] Step 2:

[0109] The server analyzes the received unstructured audio data using a natural language processing engine and converts it into text. The result is a format that allows for understanding the customer's emotions and intentions. The input is audio data, and the output is structured text data. Specifically, speech recognition technology is used to convert the audio into text, followed by text analysis to extract the user's emotions.

[0110] Step 3:

[0111] The server performs data analysis using the converted text data and purchase history. Machine learning models are used to model customer purchasing trends and preferences. The input is structured text data and purchase history, and the output is a dataset representing customer characteristics. Specifically, cluster analysis and classification algorithms are used to analyze the data.

[0112] Step 4:

[0113] The server uses a machine learning model based on data analysis results to predict future purchases. It generates specific product recommendations and formulates target products and pricing strategies. The input is customer characteristic data, and the output is a list of recommended products. The product suggestions generated by the AI ​​model are optimized by referring to prompt text.

[0114] Step 5:

[0115] The device notifies the user of the generated recommendation information and presents the most suitable products to the customer. The user can then make a purchase decision based on this information. Here, the input is the recommendation information, and the output is the notification or display to the user. Specifically, a list of recommended products is displayed on the smartphone, allowing the user to check the details.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention relates to a system that integrates and manages structured and unstructured data, performs data analysis and prediction, and recognizes user emotions, enabling the optimization of strategies based on those emotions. The system comprises processes including data ingestion, emotion recognition, data analysis, prediction, and strategy execution.

[0118] The server first receives structured and unstructured data uploaded by users and stores them in a database. Unstructured data is converted into structured data using an analysis tool, and in the process, an emotion engine is used to recognize the emotions contained in the data. This emotion recognition involves sentiment analysis from text and voice tone analysis to extract emotional characteristics such as joy, anger, and sadness.

[0119] Next, the server uses the recognized sentiment information to perform structured data analysis. This analysis employs standard statistical analysis and machine learning, while also considering the user's sentiment data. This allows for an analysis of the impact of emotional changes on business outcomes and the creation of more precise predictions.

[0120] Predictive tools make it possible to estimate future sales and market trends based on specific emotional tendencies. Based on these detailed predictions, the server provides users with information for strategic planning.

[0121] Based on the analysis results and predictions presented by the server, users formulate individual business strategies and send them to the server as commands from their terminals. The server then implements these strategies, optimized based on sentiment information, and carries out their effective execution through execution control mechanisms.

[0122] For example, in designing an advertising campaign, it's possible to consider customer emotional tendencies and develop messages with optimized content and timing to resonate emotionally. This can improve customer engagement and maximize marketing effectiveness.

[0123] Thus, the system of the present invention supports companies in quickly and effectively utilizing data and making strategic decisions through multidimensional data processing, including emotion recognition.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] Users upload structured data (e.g., sales records in spreadsheet format) and unstructured data (e.g., customer review audio files or social media text posts) to the server. The server receives this data and stores it in data storage as structured and unstructured data, respectively.

[0127] Step 2:

[0128] The server analyzes unstructured data and converts it into structured data. Speech data is converted to text using speech recognition technology, and the emotion engine recognizes customer emotions through text analysis. Emotions are categorized as positive, negative, or neutral.

[0129] Step 3:

[0130] The server performs data analysis based on the transformed structured data. This analysis includes sales records and recognized sentiment data, and utilizes machine learning models to discover correlations and customer trends. This reveals the connection between emotions and purchasing behavior.

[0131] Step 4:

[0132] The server uses predictive tools based on data analysis results to forecast future market and sales trends. It provides users with predictions that take into account the impact of emotional changes on sales. For example, it might conclude that "sales of products with many positive reviews are likely to increase by 10% next month."

[0133] Step 5:

[0134] Users develop business strategies based on these analysis results and predictions. They then send the developed strategies to the server via their device and instruct it to begin execution. For example, a user could develop a promotional plan for a new product that leverages positive reviews.

[0135] Step 6:

[0136] The server analyzes received strategic directives and develops effective implementation plans. It communicates with external systems to manage the specific deployment of advertising campaigns and promotions. The server makes adjustments based on user-specified conditions and emotional characteristics and continuously monitors the effectiveness of the campaigns.

[0137] (Example 2)

[0138] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0139] Traditional data analysis systems only handle structured data, and therefore have the limitation of not being able to effectively utilize the emotional information contained in unstructured data. As a result, it has been difficult to make predictions and develop strategies that adequately reflect customer emotional trends, leading to a lack of competitiveness, especially in the marketing domain.

[0140] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0141] In this invention, the server includes means for receiving structured and unstructured data and storing it in data storage, means for converting unstructured data into structured data and for analyzing emotions, and means for performing data analysis based on structured data while taking emotional information into consideration. This enables predictions that reflect the user's emotions and the execution of optimized strategies based on those predictions.

[0142] "Structured data" refers to information that is organized into rows and columns, like in a database or spreadsheet, and presented in a format that is easy for machines to process.

[0143] "Unstructured data" refers to information that is not organized into a specific format, such as text documents or audio files, and is difficult for machines to analyze or process.

[0144] "Data storage" refers to a physical or virtual storage device used to permanently store received data.

[0145] "Analysis means" refers to a process or technique for converting unstructured data into structured data and extracting necessary information.

[0146] "Analysis methods for recognizing emotions" refer to technologies for extracting emotional characteristics from text or audio and interpreting them.

[0147] A "processing method" is a computational process used to analyze data and interpret the results.

[0148] A "predictive tool" is a technology used to estimate future trends and outcomes based on past data and analysis results.

[0149] A "control mechanism" is a process that directs specific actions to implement a formulated strategy based on prediction results.

[0150] "Emotional information" refers to data that indicates the psychological state of the user or object, and is extracted mechanically as part of the analysis.

[0151] An "optimized strategy" is a plan that is tailored to achieve set goals in the most efficient way, based on all available information.

[0152] In this invention, the server first receives structured and unstructured data provided by the user. This data is securely stored in data storage and protected using encryption technology. Specifically, for example, a database management system is used.

[0153] Next, the server uses a natural language processing engine and speech analysis tools to convert unstructured data into a structured format. For example, common machine learning libraries are used for natural language processing. At this stage, an emotion analysis engine operates to recognize emotions and extract emotional features such as joy, anger, and sadness from the data. For example, a natural language toolkit is used for processing text data, and a speech analysis API is used for processing speech data.

[0154] Subsequently, the server performs data analysis based on structured data and sentiment information. This analysis includes methods using machine learning models to reveal past data patterns and trends. By using common machine learning frameworks, rapid and efficient data processing is achieved. This makes it possible to quantitatively understand the impact of changes in sentiment on business outcomes.

[0155] Based on the analysis results, the server uses a generative AI model to predict future sales and market trends. This prediction helps in strategy formulation and provides users with specific and useful advice. An example of a prompt might be, "Predict customer emotional responses to this week's advertising campaign and propose an effective strategy."

[0156] Finally, users develop customized strategies on their devices based on predictive information supported by data provided by the server. These strategies are then returned to the server, optimized to incorporate emotional information, and finally implemented. As a result, companies can make strategic decisions based on customer emotions, leading to more effective marketing and enhanced customer relationship management.

[0157] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0158] Step 1:

[0159] The server receives structured and unstructured data from users and stores them in data storage. Input is data files uploaded by users, and output is data stored in storage. The received data is protected using encryption technology to ensure secure access.

[0160] Step 2:

[0161] The server uses a natural language processing engine and speech analysis tools to convert unstructured data into structured format. The input is unstructured data (e.g., text documents or audio files), and the output is analyzed structured data. This conversion process includes extracting keywords from text and audio and converting them into standardized data formats.

[0162] Step 3:

[0163] The server activates the sentiment analysis engine and recognizes sentiment information in the structured data. The input is the structured data obtained in step 2, and the output is the expanded structured data containing sentiment features. Specifically, this involves using a text analysis algorithm to extract sentiment features such as joy, anger, and sadness.

[0164] Step 4:

[0165] The server performs data analysis while considering emotional information. The input is structured data containing emotional features, and the output is the analysis results. In this step, a machine learning model is used to reveal business trends and patterns by relating past data analysis with the influence of emotions.

[0166] Step 5:

[0167] The server uses a generative AI model based on the analysis results to predict future sales and market trends. The input is the data analysis results, and the output is a prediction report. In this step, deep learning techniques are used to make more precise predictions, and the generative AI model generates prompts on how to respond.

[0168] Step 6:

[0169] Users develop business strategies on their devices based on forecast reports provided by the server. The input is the forecast report, and the output is the formulated strategic directive. In this process, users plan advertising campaigns and sales strategies based on the analyzed data.

[0170] Step 7:

[0171] The server receives strategic commands from the user, utilizes emotional information to optimize and execute the strategy. The input is the strategic command from the user, and the output is the optimized strategy that has been executed. Specific actions include adjusting the appropriate timing and content of ad delivery.

[0172] (Application Example 2)

[0173] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0174] In modern communication, information delivery that takes user emotions into consideration is extremely limited. Existing advertising and information delivery systems are primarily based on users' past behavioral history and attribute information, and there are virtually no real-time means of delivering information that takes into account the user's current emotional state. As a result, users may not receive information that is truly valuable to them. There is a need to improve this situation and realize means of delivering information that accurately captures users' emotions.

[0175] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0176] In this invention, the server includes means for receiving structured and unstructured data and storing them in data storage, means for converting unstructured data into structured data and for extracting emotional characteristics, and means for processing information based on emotional characteristics in real time. This makes it possible to analyze the user's emotional state in real time and provide optimal information accordingly.

[0177] "Structured data" refers to data that is stored in a formatted form, such as in a database, and can be easily searched and analyzed.

[0178] "Unstructured data" refers to data that does not have a specific format and exists in a wide variety of forms, such as text, images, and audio.

[0179] "Data storage" refers to a platform for storing digital data, including storage systems on servers and in the cloud.

[0180] "Analysis methods" refer to the processes and tools used to analyze unstructured data and convert it into structured data.

[0181] "Emotional features" refer to characteristics that indicate human emotional states such as joy, sadness, and anger, extracted from text or audio.

[0182] "Processing means" refers to computer programs or algorithms used to analyze data and generate specific outputs related to the user.

[0183] The system of the present invention enables real-time information provision by integrating structured and unstructured data and performing user emotion recognition based on these. The server first receives the user's structured and unstructured data and stores it in data storage. Unstructured data is converted into structured data using analysis means, and emotion features are extracted. This process uses software that implements algorithms for natural language processing and emotion analysis. Specifically, the NLP library TextBlob can be used.

[0184] After emotional characteristics are extracted, the server uses processing tools to generate information tailored to the user's emotional state, such as advertisements and related information, in real time and provide it to the user's terminal. This provides optimal information that fits the user's emotions, improving the relevance and value of the information.

[0185] As a concrete example, if a user posts "It was a fun day" on social media, the server could use TextBlob to extract positive emotions and provide promotional information about recreational facilities. This system can be implemented as a Python®-based application and utilizes a generative AI model to perform accelerated sentiment analysis.

[0186] An example of a prompt could be: "Analyze the sentiment of the text and suggest an advertising message based on that sentiment."

[0187] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0188] Step 1:

[0189] The server stores structured and unstructured data received from users in data storage. Input is user text and audio data, and output is the retention of this data in storage. The stored data forms the basis for subsequent processing.

[0190] Step 2:

[0191] The server receives unstructured data and converts it into structured data using analysis tools. This process uses natural language processing libraries such as TextBlob to perform sentiment analysis on text data and extract emotional features. The input is unstructured text data, and the output is structured data including sentiment tags (e.g., positive, negative, neutral).

[0192] Step 3:

[0193] The server generates user-specific information based on sentiment characteristics extracted from processed data. Here, it retrieves advertisements and information corresponding to predefined sentiment tags from a database. The input consists of sentiment characteristics and historical advertising data, while the output is optimized advertising information. This allows for real-time selection of information tailored to the user's emotional state.

[0194] Step 4:

[0195] The device displays information received from the server on the user interface. During display, the visuals are adjusted to match the user's emotions and presented in an appealing format. The input is optimized advertising information, and the output is the display of advertisements to the user. This process allows the user to receive information that aligns with their mood.

[0196] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0197] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0198] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0199] [Second Embodiment]

[0200] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0201] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0202] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0203] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0204] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0205] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0206] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0207] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0208] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0209] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0210] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0211] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0212] This invention relates to a system for integrating and managing structured and unstructured data and executing truly useful business strategies through data analysis and forecasting. The system includes processes for data ingestion, analysis of unstructured data, data analysis, forecasting, and strategy execution.

[0213] The server is designed to receive both structured and unstructured data and stores this data in data storage. Unstructured data is converted into structured data using specific analytical methods. Through this analysis, the unstructured data is formatted into a format that is easily visualized and measured using techniques such as natural language processing and image recognition.

[0214] Next, the server performs data analysis using the analyzed structured data. This is a process to extract useful information and trends from past data. The insights gained through this analysis process are used to estimate future outcomes using predictive tools. For example, this can be used to forecast sales or customer behavior.

[0215] Users can formulate specific business strategies based on analysis results and predictions provided by the server. They can then send the selected strategy to the system via their terminal to initiate its execution. The server receives the command to execute the strategy and implements it effectively through its control mechanisms.

[0216] As a concrete example, consider a scenario where a user uploads unstructured audio data of customer feedback to the system. The server converts this audio data into text and uses natural language processing to analyze the customer's emotions and intentions. Based on these results, the server predicts increases or decreases in sales and the need for product improvements. The user then develops a new marketing campaign based on these predictions and sends execution commands through the system. The server manages the resources necessary to run the campaign and deploys it at the optimal time.

[0217] Thus, the system of the present invention enables companies to comprehensively utilize data and make quick and effective decisions.

[0218] The following describes the processing flow.

[0219] Step 1:

[0220] Users upload structured data (e.g., database export files) and unstructured data (e.g., audio files and text documents) to the server. The server receives these files and stores the structured data in the `structured_data` list and the unstructured data in the `unstructured_data` list.

[0221] Step 2:

[0222] The server initiates a process to analyze unstructured data and convert it into structured data. For example, after converting audio data to text, it uses natural language processing to extract keywords and perform sentiment analysis from the text. The server then adds the results of this conversion to the structured_data list as structured information.

[0223] Step 3:

[0224] The server performs data analysis based on structured data lists. This analysis includes applying statistical analysis and machine learning algorithms to gain useful insights from the data. For example, it identifies best-selling products and seasonal trends based on historical sales data.

[0225] Step 4:

[0226] The server makes future predictions based on the results of data analysis. This includes using predictive models to forecast future sales and estimate market trends. The prediction results are provided to users to help them in strategic planning.

[0227] Step 5:

[0228] The user develops a specific business strategy based on the prediction results from the server. The strategy is then sent to the server as a command using a terminal, initiating the strategy's execution process.

[0229] Step 6:

[0230] The server executes the plan based on the strategic directives it receives. This execution includes managing the optimal deployment of campaigns and promotions while coordinating with external systems. The server monitors overall progress and makes adjustments to ensure success.

[0231] (Example 1)

[0232] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0233] Traditional data management systems have faced challenges in effectively integrating structured and unstructured information and quickly reflecting it in actual business strategies. Unstructured information is diverse in format and difficult to analyze, resulting in significant time and effort required to gain practical insights. Furthermore, there has been a lack of efficient means to execute strategies based on future predictions.

[0234] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0235] In this invention, the server includes means for receiving structured and unstructured information and storing it in information memory; means for analyzing unstructured information to convert it into structured information; means for processing information based on the structured information; means for analyzing unstructured information using natural language processing or image recognition technology; and means for transmitting policy execution commands through a terminal. This enables rapid decision-making and effective implementation of strategies by comprehensively utilizing structured and unstructured information.

[0236] "Structured information" refers to information organized according to a specific format or structure, and is typically stored in databases, spreadsheets, and other similar formats.

[0237] "Unstructured information" refers to information that does not adhere to a specific format and exists in a variety of forms, including audio data, image data, and text documents.

[0238] "Information storage" refers to physical or digital storage devices for holding structured and unstructured information.

[0239] "Information analysis means" refers to technologies and tools that have the function of analyzing unstructured information and converting it into structured information.

[0240] "Processing means for performing information analysis" refers to technologies that have the function of performing calculations and models to derive trends and patterns based on structured information.

[0241] "Natural language processing" refers to the technology that enables computers to understand, interpret, and generate human language.

[0242] "Image recognition technology" refers to the technology used to analyze image data and recognize specific objects or features.

[0243] A "terminal" refers to a digital device used by a user to communicate with a server and perform operations or transmit information.

[0244] A "policy execution command" refers to an instruction that prompts a server to begin carrying out specific actions or procedures.

[0245] This invention is a system that enables companies to integrate and utilize structured and unstructured information to formulate and execute useful business strategies. Specific embodiments are described below.

[0246] The server performs its primary functions, primarily information management. First, the server receives structured and unstructured information from users and stores it in its information storage device. Structured information is retrieved from existing databases, while unstructured information includes various data formats such as audio, images, and text. This unstructured information is analyzed by the server using natural language processing and image recognition technologies and converted into structured information. This process utilizes application-specific analysis software.

[0247] The analyzed information is processed on an information analysis platform within the server. This platform uses advanced machine learning algorithms, including generative AI models, to extract customer behavior patterns and market trends from past data. Based on these results, the server predicts future outcomes and provides them to the user.

[0248] Users receive analysis results and predictions from the server via their terminals and formulate new business strategies based on them. In this process, users send instructions from their terminals to the server to prompt specific actions.

[0249] As a concrete example, when a user uploads audio data of customer feedback to a server, the server converts the data into text and performs a sentiment analysis of the customer through natural language processing. Based on the insights gained from this analysis, the user can decide on strategies for increasing sales or improving products. They then send these strategies to the server as execution commands, and the server carries out campaigns and operations according to those commands.

[0250] An example of a prompt for a generative AI model is, "Explain the process of converting audio data into text and using that content to make sales forecasts." This system supports companies in making quick and effective decisions by utilizing information from multiple perspectives.

[0251] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0252] Step 1:

[0253] Users upload structured and unstructured information to the server using their devices. Unstructured information includes customer voice data and email text. This information is stored in a data storage device by the server. Inputs are in various data formats, and these data are securely recorded as outputs.

[0254] Step 2:

[0255] The server uses natural language processing and image recognition technologies to analyze unstructured information. Specifically, it converts audio data into text and uses natural language processing tools to analyze customer emotions and intentions. The input is uploaded unstructured information, and the output is analyzed structured information.

[0256] Step 3:

[0257] The server performs information analysis based on structured information. This includes processes that utilize machine learning algorithms, such as generative AI models, to derive useful trends and patterns from past data. The input is structured information, and the output is the analyzed insights and patterns.

[0258] Step 4:

[0259] The server uses prediction tools based on the analysis results to estimate future outcomes. It performs simulations of sales trends and customer behavior, quantifying future prospects. The input is the analyzed information, and the output is the predicted future results.

[0260] Step 5:

[0261] Users receive prediction results from the server and formulate new business strategies based on the information displayed on their devices. These strategies are then reflected in specific policies, such as the planning of more proactive marketing campaigns. The input is the prediction results, and the output is the formulated strategy.

[0262] Step 6:

[0263] The user sends the formulated strategy to the server as an execution command via their terminal. The server receives this command and begins implementing the policy using its control mechanisms. The input is the command from the user, and the output is the specific actions taken to implement it.

[0264] (Application Example 1)

[0265] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] In today's business environment, it is crucial to extract useful information from diverse data and formulate and execute concrete strategies. However, in situations where structured and unstructured data coexist, efficiently analyzing them and providing customized proposals for individual customers is difficult. Therefore, companies are required to utilize data quickly and effectively to enhance their competitiveness.

[0267] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0268] In this invention, the server includes means for receiving structured and unstructured data and storing it in a data storage device, analysis means for converting unstructured data into structured data, and processing means for performing data analysis based on the structured data. This makes it possible to propose products optimized for the customer.

[0269] "Structured data" refers to data that exists in a well-organized, structured format, such as in databases or spreadsheets, making it easy to search and analyze.

[0270] "Unstructured data" refers to data in formats that do not have a specific structure, such as audio, text, and image data, and is difficult to analyze in its original form.

[0271] A "data storage device" is a physical or virtual storage device used to store received data.

[0272] "Analysis methods" refer to the process of processing data using technologies such as natural language processing and image recognition in order to convert unstructured data into structured data.

[0273] "Processing means" refers to a function that uses structured data to perform statistical analysis and machine learning-based data analysis to extract useful information.

[0274] A "predictive function" is a technology that estimates future trends and outcomes based on analyzed data and generates prediction results.

[0275] The "control function" is the process of issuing commands to execute strategies formulated based on prediction results and managing their progress.

[0276] The "suggestion function" is a system that provides optimized product information and purchase suggestions based on customer history and feedback.

[0277] A "communication function" is an interface for exchanging data with an external information processing device.

[0278] The system of the present invention integrates the functions of data reception, analysis, interpretation, prediction, proposal, and execution. The server receives structured and unstructured data and stores it in a data storage device. Unstructured data is converted into structured data using natural language processing and image recognition technologies, and this is carried out through analysis means. The technologies used can be combined with conventional database management systems and include NLP engines and image recognition software.

[0279] The server performs data analysis using the transformed structured data and makes future predictions using machine learning models based on the features extracted from the data. Machine learning frameworks such as TensorFlow and PyTorch are utilized for this purpose. For example, it can recommend the most suitable products and services for the future based on customer purchase history and feedback.

[0280] On the terminal, the generated AI model is operated according to instructions from the user, a strategy based on the prediction results is formulated, and a command for execution is sent to the server. As an example of the prompt sentence, it is implemented in the form of "Analyze the customer's past feedback and then generate a list of the products most likely to be purchased. Pay particular attention to feedback on size and product color from unstructured voice data."

[0281] As a result, it can be utilized as information when conducting new marketing campaigns or optimizing price settings, enabling flexible and prompt responses based on customer needs in the enterprise.

[0282] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0283] Step 1:

[0284] The server receives customer feedback (unstructured data) and purchase history (structured data) in the form of voice or text from the smartphone terminal. The received data is stored in the data storage device. Here, the input is voice data and past purchase history, and the output is the raw data stored in the data storage device.

[0285] Step 2:

[0286] The server analyzes the received unstructured voice data using a natural language processing engine and converts it into text. As a result, it is in a form that enables understanding of the customer's emotions and intentions. The input is voice data, and the output is structured text data. Specifically, the voice is converted into text by voice recognition technology, and then text analysis is performed to extract the user's emotions.

[0287] Step 3:

[0288] The server performs data analysis using the converted text data and purchase history. Machine learning models are used to model customer purchasing trends and preferences. The input is structured text data and purchase history, and the output is a dataset representing customer characteristics. Specifically, cluster analysis and classification algorithms are used to analyze the data.

[0289] Step 4:

[0290] The server uses a machine learning model based on data analysis results to predict future purchases. It generates specific product recommendations and formulates target products and pricing strategies. The input is customer characteristic data, and the output is a list of recommended products. The product suggestions generated by the AI ​​model are optimized by referring to prompt text.

[0291] Step 5:

[0292] The device notifies the user of the generated recommendation information and presents the most suitable products to the customer. The user can then make a purchase decision based on this information. Here, the input is the recommendation information, and the output is the notification or display to the user. Specifically, a list of recommended products is displayed on the smartphone, allowing the user to check the details.

[0293] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0294] This invention relates to a system that integrates and manages structured and unstructured data, performs data analysis and prediction, and recognizes user emotions, enabling the optimization of strategies based on those emotions. The system comprises processes including data ingestion, emotion recognition, data analysis, prediction, and strategy execution.

[0295] The server first receives structured and unstructured data uploaded by users and stores them in a database. Unstructured data is converted into structured data using an analysis tool, and in the process, an emotion engine is used to recognize the emotions contained in the data. This emotion recognition involves sentiment analysis from text and voice tone analysis to extract emotional characteristics such as joy, anger, and sadness.

[0296] Next, the server uses the recognized sentiment information to perform structured data analysis. This analysis employs standard statistical analysis and machine learning, while also considering the user's sentiment data. This allows for an analysis of the impact of emotional changes on business outcomes and the creation of more precise predictions.

[0297] Predictive tools make it possible to estimate future sales and market trends based on specific emotional tendencies. Based on these detailed predictions, the server provides users with information for strategic planning.

[0298] Based on the analysis results and predictions presented by the server, users formulate individual business strategies and send them to the server as commands from their terminals. The server then implements these strategies, optimized based on sentiment information, and carries out their effective execution through execution control mechanisms.

[0299] For example, in designing an advertising campaign, it's possible to consider customer emotional tendencies and develop messages with optimized content and timing to resonate emotionally. This can improve customer engagement and maximize marketing effectiveness.

[0300] Thus, the system of the present invention supports companies in quickly and effectively utilizing data and making strategic decisions through multidimensional data processing, including emotion recognition.

[0301] The following describes the processing flow.

[0302] Step 1:

[0303] The user uploads structured data (e.g., sales records in spreadsheet format) and unstructured data (e.g., customer review voice files and SNS text posts) to the server. The server receives these data and stores them in the data storage as structured data and unstructured data respectively.

[0304] Step 2:

[0305] The server analyzes the unstructured data and converts it into structured data. For voice data, voice recognition technology is used to convert it into text, and through text analysis, the sentiment engine recognizes the customer's sentiment. The sentiment is classified into categories such as positive, negative, and neutral.

[0306] Step 3:

[0307] The server performs data analysis based on the converted structured data. For data analysis, a machine learning model for discovering correlations and customer trends is used, including sales records and recognized sentiment data. This reveals the connection between sentiment and purchasing behavior.

[0308] Step 4:

[0309] The server uses prediction means based on the results of data analysis to predict future market trends and sales trends. A prediction that takes into account the impact of sentiment changes on sales is made and provided to the user. For example, a conclusion such as "Sales of products with many positive reviews may increase by 10% in the next month" is derived.

[0310] Step 5:

[0311] The user formulates a business strategy based on these analysis results and predictions. Through the terminal, the formulated strategy is sent to the server and an instruction to start execution is given. The user can, for example, formulate a promotion plan for a new product that makes use of positive reviews.

[0312] Step 6:

[0313] The server analyzes received strategic directives and develops effective implementation plans. It communicates with external systems to manage the specific deployment of advertising campaigns and promotions. The server makes adjustments based on user-specified conditions and emotional characteristics and continuously monitors the effectiveness of the campaigns.

[0314] (Example 2)

[0315] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0316] Traditional data analysis systems only handle structured data, and therefore have the limitation of not being able to effectively utilize the emotional information contained in unstructured data. As a result, it has been difficult to make predictions and develop strategies that adequately reflect customer emotional trends, leading to a lack of competitiveness, especially in the marketing domain.

[0317] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0318] In this invention, the server includes means for receiving structured and unstructured data and storing it in data storage, means for converting unstructured data into structured data and for analyzing emotions, and means for performing data analysis based on structured data while taking emotional information into consideration. This enables predictions that reflect the user's emotions and the execution of optimized strategies based on those predictions.

[0319] "Structured data" refers to information that is organized into rows and columns, like in a database or spreadsheet, and presented in a format that is easy for machines to process.

[0320] "Unstructured data" refers to information that is not organized into a specific format, such as text documents or audio files, and is difficult for machines to analyze or process.

[0321] "Data storage" refers to a physical or virtual storage device used to permanently store received data.

[0322] "Analysis means" refers to a process or technique for converting unstructured data into structured data and extracting necessary information.

[0323] "Analysis methods for recognizing emotions" refer to technologies for extracting emotional characteristics from text or audio and interpreting them.

[0324] A "processing method" is a computational process used to analyze data and interpret the results.

[0325] A "predictive tool" is a technology used to estimate future trends and outcomes based on past data and analysis results.

[0326] A "control mechanism" is a process that directs specific actions to implement a formulated strategy based on prediction results.

[0327] "Emotional information" refers to data that indicates the psychological state of the user or object, and is extracted mechanically as part of the analysis.

[0328] An "optimized strategy" is a plan that is tailored to achieve set goals in the most efficient way, based on all available information.

[0329] In this invention, the server first receives structured and unstructured data provided by the user. This data is securely stored in data storage and protected using encryption technology. Specifically, for example, a database management system is used.

[0330] Next, the server uses a natural language processing engine and speech analysis tools to convert unstructured data into a structured format. For example, common machine learning libraries are used for natural language processing. At this stage, an emotion analysis engine operates to recognize emotions and extract emotional features such as joy, anger, and sadness from the data. For example, a natural language toolkit is used for processing text data, and a speech analysis API is used for processing speech data.

[0331] Subsequently, the server performs data analysis based on structured data and sentiment information. This analysis includes methods using machine learning models to reveal past data patterns and trends. By using common machine learning frameworks, rapid and efficient data processing is achieved. This makes it possible to quantitatively understand the impact of changes in sentiment on business outcomes.

[0332] Based on the analysis results, the server uses a generative AI model to predict future sales and market trends. This prediction helps in strategy formulation and provides users with specific and useful advice. An example of a prompt might be, "Predict customer emotional responses to this week's advertising campaign and propose an effective strategy."

[0333] Finally, users develop customized strategies on their devices based on predictive information supported by data provided by the server. These strategies are then returned to the server, optimized to incorporate emotional information, and finally implemented. As a result, companies can make strategic decisions based on customer emotions, leading to more effective marketing and enhanced customer relationship management.

[0334] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0335] Step 1:

[0336] The server receives structured and unstructured data from users and stores them in data storage. Input is data files uploaded by users, and output is data stored in storage. The received data is protected using encryption technology to ensure secure access.

[0337] Step 2:

[0338] The server uses a natural language processing engine and speech analysis tools to convert unstructured data into structured format. The input is unstructured data (e.g., text documents or audio files), and the output is analyzed structured data. This conversion process includes extracting keywords from text and audio and converting them into standardized data formats.

[0339] Step 3:

[0340] The server activates the sentiment analysis engine and recognizes sentiment information in the structured data. The input is the structured data obtained in step 2, and the output is the expanded structured data containing sentiment features. Specifically, this involves using a text analysis algorithm to extract sentiment features such as joy, anger, and sadness.

[0341] Step 4:

[0342] The server performs data analysis while considering emotional information. The input is structured data containing emotional features, and the output is the analysis results. In this step, a machine learning model is used to reveal business trends and patterns by relating past data analysis with the influence of emotions.

[0343] Step 5:

[0344] The server uses a generative AI model based on the analysis results to predict future sales and market trends. The input is the data analysis results, and the output is a prediction report. In this step, deep learning techniques are used to make more precise predictions, and the generative AI model generates prompts on how to respond.

[0345] Step 6:

[0346] Users develop business strategies on their devices based on forecast reports provided by the server. The input is the forecast report, and the output is the formulated strategic directive. In this process, users plan advertising campaigns and sales strategies based on the analyzed data.

[0347] Step 7:

[0348] The server receives strategic commands from the user, utilizes emotional information to optimize and execute the strategy. The input is the strategic command from the user, and the output is the optimized strategy that has been executed. Specific actions include adjusting the appropriate timing and content of ad delivery.

[0349] (Application Example 2)

[0350] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0351] In modern communication, information delivery that takes user emotions into consideration is extremely limited. Existing advertising and information delivery systems are primarily based on users' past behavioral history and attribute information, and there are virtually no real-time means of delivering information that takes into account the user's current emotional state. As a result, users may not receive information that is truly valuable to them. There is a need to improve this situation and realize means of delivering information that accurately captures users' emotions.

[0352] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0353] In this invention, the server includes means for receiving structured and unstructured data and storing them in data storage, means for converting unstructured data into structured data and for extracting emotional characteristics, and means for processing information based on emotional characteristics in real time. This makes it possible to analyze the user's emotional state in real time and provide optimal information accordingly.

[0354] "Structured data" refers to data that is stored in a formatted form, such as in a database, and can be easily searched and analyzed.

[0355] "Unstructured data" refers to data that does not have a specific format and exists in a wide variety of forms, such as text, images, and audio.

[0356] "Data storage" refers to a platform for storing digital data, including storage systems on servers and in the cloud.

[0357] "Analysis methods" refer to the processes and tools used to analyze unstructured data and convert it into structured data.

[0358] "Emotional features" refer to characteristics that indicate human emotional states such as joy, sadness, and anger, extracted from text or audio.

[0359] "Processing means" refers to computer programs or algorithms used to analyze data and generate specific outputs related to the user.

[0360] The system of the present invention enables real-time information provision by integrating structured and unstructured data and performing user emotion recognition based on these. The server first receives the user's structured and unstructured data and stores it in data storage. Unstructured data is converted into structured data using analysis means, and emotion features are extracted. This process uses software that implements algorithms for natural language processing and emotion analysis. Specifically, the NLP library TextBlob can be used.

[0361] After emotional characteristics are extracted, the server uses processing tools to generate information tailored to the user's emotional state, such as advertisements and related information, in real time and provide it to the user's terminal. This provides optimal information that fits the user's emotions, improving the relevance and value of the information.

[0362] As a concrete example, if a user posts "It was a fun day" on social media, the server could use TextBlob to extract positive emotions and provide promotional information about recreational facilities. This system can be implemented as a Python-based application and utilizes a generative AI model to perform accelerated sentiment analysis.

[0363] An example of a prompt could be: "Analyze the sentiment of the text and suggest an advertising message based on that sentiment."

[0364] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0365] Step 1:

[0366] The server stores structured and unstructured data received from users in data storage. Input is user text and audio data, and output is the retention of this data in storage. The stored data forms the basis for subsequent processing.

[0367] Step 2:

[0368] The server receives unstructured data and converts it into structured data using analysis tools. This process uses natural language processing libraries such as TextBlob to perform sentiment analysis on text data and extract emotional features. The input is unstructured text data, and the output is structured data including sentiment tags (e.g., positive, negative, neutral).

[0369] Step 3:

[0370] The server generates user-specific information based on sentiment characteristics extracted from processed data. Here, it retrieves advertisements and information corresponding to predefined sentiment tags from a database. The input consists of sentiment characteristics and historical advertising data, while the output is optimized advertising information. This allows for real-time selection of information tailored to the user's emotional state.

[0371] Step 4:

[0372] The device displays information received from the server on the user interface. During display, the visuals are adjusted to match the user's emotions and presented in an appealing format. The input is optimized advertising information, and the output is the display of advertisements to the user. This process allows the user to receive information that aligns with their mood.

[0373] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0374] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0375] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0376] [Third Embodiment]

[0377] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0378] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0379] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0380] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0381] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0382] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0383] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0384] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0385] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0386] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0387] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0388] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0389] This invention relates to a system for integrating and managing structured and unstructured data and executing truly useful business strategies through data analysis and forecasting. The system includes processes for data ingestion, analysis of unstructured data, data analysis, forecasting, and strategy execution.

[0390] The server is designed to receive both structured and unstructured data and stores this data in data storage. Unstructured data is converted into structured data using specific analytical methods. Through this analysis, the unstructured data is formatted into a format that is easily visualized and measured using techniques such as natural language processing and image recognition.

[0391] Next, the server performs data analysis using the analyzed structured data. This is a process to extract useful information and trends from past data. The insights gained through this analysis process are used to estimate future outcomes using predictive tools. For example, this can be used to forecast sales or customer behavior.

[0392] Users can formulate specific business strategies based on analysis results and predictions provided by the server. They can then send the selected strategy to the system via their terminal to initiate its execution. The server receives the command to execute the strategy and implements it effectively through its control mechanisms.

[0393] As a concrete example, consider a scenario where a user uploads unstructured audio data of customer feedback to the system. The server converts this audio data into text and uses natural language processing to analyze the customer's emotions and intentions. Based on these results, the server predicts increases or decreases in sales and the need for product improvements. The user then develops a new marketing campaign based on these predictions and sends execution commands through the system. The server manages the resources necessary to run the campaign and deploys it at the optimal time.

[0394] Thus, the system of the present invention enables companies to comprehensively utilize data and make quick and effective decisions.

[0395] The following describes the processing flow.

[0396] Step 1:

[0397] Users upload structured data (e.g., database export files) and unstructured data (e.g., audio files and text documents) to the server. The server receives these files and stores the structured data in the `structured_data` list and the unstructured data in the `unstructured_data` list.

[0398] Step 2:

[0399] The server initiates a process to analyze unstructured data and convert it into structured data. For example, after converting audio data to text, it uses natural language processing to extract keywords and perform sentiment analysis from the text. The server then adds the results of this conversion to the structured_data list as structured information.

[0400] Step 3:

[0401] The server performs data analysis based on structured data lists. This analysis includes applying statistical analysis and machine learning algorithms to gain useful insights from the data. For example, it identifies best-selling products and seasonal trends based on historical sales data.

[0402] Step 4:

[0403] The server makes future predictions based on the results of data analysis. This includes using predictive models to forecast future sales and estimate market trends. The prediction results are provided to users to help them in strategic planning.

[0404] Step 5:

[0405] The user develops a specific business strategy based on the prediction results from the server. The strategy is then sent to the server as a command using a terminal, initiating the strategy's execution process.

[0406] Step 6:

[0407] The server executes the plan based on the strategic directives it receives. This execution includes managing the optimal deployment of campaigns and promotions while coordinating with external systems. The server monitors overall progress and makes adjustments to ensure success.

[0408] (Example 1)

[0409] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0410] Traditional data management systems have faced challenges in effectively integrating structured and unstructured information and quickly reflecting it in actual business strategies. Unstructured information is diverse in format and difficult to analyze, resulting in significant time and effort required to gain practical insights. Furthermore, there has been a lack of efficient means to execute strategies based on future predictions.

[0411] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0412] In this invention, the server includes means for receiving structured and unstructured information and storing it in information memory; means for analyzing unstructured information to convert it into structured information; means for processing information based on the structured information; means for analyzing unstructured information using natural language processing or image recognition technology; and means for transmitting policy execution commands through a terminal. This enables rapid decision-making and effective implementation of strategies by comprehensively utilizing structured and unstructured information.

[0413] "Structured information" refers to information organized according to a specific format or structure, and is typically stored in databases, spreadsheets, and other similar formats.

[0414] "Unstructured information" refers to information that does not adhere to a specific format and exists in a variety of forms, including audio data, image data, and text documents.

[0415] "Information storage" refers to physical or digital storage devices for holding structured and unstructured information.

[0416] "Information analysis means" refers to technologies and tools that have the function of analyzing unstructured information and converting it into structured information.

[0417] "Processing means for performing information analysis" refers to technologies that have the function of performing calculations and models to derive trends and patterns based on structured information.

[0418] "Natural language processing" refers to the technology that enables computers to understand, interpret, and generate human language.

[0419] "Image recognition technology" refers to the technology used to analyze image data and recognize specific objects or features.

[0420] A "terminal" refers to a digital device used by a user to communicate with a server and perform operations or transmit information.

[0421] A "policy execution command" refers to an instruction that prompts a server to begin carrying out specific actions or procedures.

[0422] This invention is a system that enables companies to integrate and utilize structured and unstructured information to formulate and execute useful business strategies. Specific embodiments are described below.

[0423] The server performs its primary functions, primarily information management. First, the server receives structured and unstructured information from users and stores it in its information storage device. Structured information is retrieved from existing databases, while unstructured information includes various data formats such as audio, images, and text. This unstructured information is analyzed by the server using natural language processing and image recognition technologies and converted into structured information. This process utilizes application-specific analysis software.

[0424] The analyzed information is processed on an information analysis platform within the server. This platform uses advanced machine learning algorithms, including generative AI models, to extract customer behavior patterns and market trends from past data. Based on these results, the server predicts future outcomes and provides them to the user.

[0425] Users receive analysis results and predictions from the server via their terminals and formulate new business strategies based on them. In this process, users send instructions from their terminals to the server to prompt specific actions.

[0426] As a concrete example, when a user uploads audio data of customer feedback to a server, the server converts the data into text and performs a sentiment analysis of the customer through natural language processing. Based on the insights gained from this analysis, the user can decide on strategies for increasing sales or improving products. They then send these strategies to the server as execution commands, and the server carries out campaigns and operations according to those commands.

[0427] An example of a prompt for a generative AI model is, "Explain the process of converting audio data into text and using that content to make sales forecasts." This system supports companies in making quick and effective decisions by utilizing information from multiple perspectives.

[0428] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0429] Step 1:

[0430] Users upload structured and unstructured information to the server using their devices. Unstructured information includes customer voice data and email text. This information is stored in a data storage device by the server. Inputs are in various data formats, and these data are securely recorded as outputs.

[0431] Step 2:

[0432] The server uses natural language processing and image recognition technologies to analyze unstructured information. Specifically, it converts audio data into text and uses natural language processing tools to analyze customer emotions and intentions. The input is uploaded unstructured information, and the output is analyzed structured information.

[0433] Step 3:

[0434] The server performs information analysis based on structured information. This includes processes that utilize machine learning algorithms, such as generative AI models, to derive useful trends and patterns from past data. The input is structured information, and the output is the analyzed insights and patterns.

[0435] Step 4:

[0436] The server uses prediction tools based on the analysis results to estimate future outcomes. It performs simulations of sales trends and customer behavior, quantifying future prospects. The input is the analyzed information, and the output is the predicted future results.

[0437] Step 5:

[0438] Users receive prediction results from the server and formulate new business strategies based on the information displayed on their devices. These strategies are then reflected in specific policies, such as the planning of more proactive marketing campaigns. The input is the prediction results, and the output is the formulated strategy.

[0439] Step 6:

[0440] The user sends the formulated strategy to the server as an execution command via their terminal. The server receives this command and begins implementing the policy using its control mechanisms. The input is the command from the user, and the output is the specific actions taken to implement it.

[0441] (Application Example 1)

[0442] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0443] In today's business environment, it is crucial to extract useful information from diverse data and formulate and execute concrete strategies. However, in situations where structured and unstructured data coexist, efficiently analyzing them and providing customized proposals for individual customers is difficult. Therefore, companies are required to utilize data quickly and effectively to enhance their competitiveness.

[0444] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0445] In this invention, the server includes means for receiving structured and unstructured data and storing it in a data storage device, analysis means for converting unstructured data into structured data, and processing means for performing data analysis based on the structured data. This makes it possible to propose products optimized for the customer.

[0446] "Structured data" refers to data that exists in a well-organized, structured format, such as in databases or spreadsheets, making it easy to search and analyze.

[0447] "Unstructured data" refers to data in formats that do not have a specific structure, such as audio, text, and image data, and is difficult to analyze in its original form.

[0448] A "data storage device" is a physical or virtual storage device used to store received data.

[0449] "Analysis methods" refer to the process of processing data using technologies such as natural language processing and image recognition in order to convert unstructured data into structured data.

[0450] "Processing means" refers to a function that uses structured data to perform statistical analysis and machine learning-based data analysis to extract useful information.

[0451] A "predictive function" is a technology that estimates future trends and outcomes based on analyzed data and generates prediction results.

[0452] The "control function" is the process of issuing commands to execute strategies formulated based on prediction results and managing their progress.

[0453] The "suggestion function" is a system that provides optimized product information and purchase suggestions based on customer history and feedback.

[0454] A "communication function" is an interface for exchanging data with an external information processing device.

[0455] The system of the present invention integrates the functions of data reception, analysis, interpretation, prediction, proposal, and execution. The server receives structured and unstructured data and stores it in a data storage device. Unstructured data is converted into structured data using natural language processing and image recognition technologies, and this is carried out through analysis means. The technologies used can be combined with conventional database management systems and include NLP engines and image recognition software.

[0456] The server performs data analysis using the transformed structured data and makes future predictions using machine learning models based on the features extracted from the data. Machine learning frameworks such as TensorFlow and PyTorch are utilized for this purpose. For example, it can recommend the most suitable products and services for the future based on customer purchase history and feedback.

[0457] The terminal operates a generating AI model based on user instructions, formulates a strategy based on the prediction results, and sends commands to the server for execution. An example of a prompt might be: "Analyze past customer feedback and generate a list of products they are most likely to purchase next. Pay particular attention to feedback regarding sizing and product color from unstructured voice data."

[0458] As a result, this information can be used to develop new marketing campaigns and optimize pricing, enabling companies to respond flexibly and quickly to customer needs.

[0459] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0460] Step 1:

[0461] The server receives customer feedback (unstructured data) in the form of voice and text, as well as purchase history (structured data), from smartphone devices. The received data is stored in a data storage device. Here, the input is voice data and past purchase history, and the output is the raw data stored in the data storage device.

[0462] Step 2:

[0463] The server analyzes the received unstructured audio data using a natural language processing engine and converts it into text. The result is a format that allows for understanding the customer's emotions and intentions. The input is audio data, and the output is structured text data. Specifically, speech recognition technology is used to convert the audio into text, followed by text analysis to extract the user's emotions.

[0464] Step 3:

[0465] The server performs data analysis using the converted text data and purchase history. Machine learning models are used to model customer purchasing trends and preferences. The input is structured text data and purchase history, and the output is a dataset representing customer characteristics. Specifically, cluster analysis and classification algorithms are used to analyze the data.

[0466] Step 4:

[0467] The server uses a machine learning model based on data analysis results to predict future purchases. It generates specific product recommendations and formulates target products and pricing strategies. The input is customer characteristic data, and the output is a list of recommended products. The product suggestions generated by the AI ​​model are optimized by referring to prompt text.

[0468] Step 5:

[0469] The device notifies the user of the generated recommendation information and presents the most suitable products to the customer. The user can then make a purchase decision based on this information. Here, the input is the recommendation information, and the output is the notification or display to the user. Specifically, a list of recommended products is displayed on the smartphone, allowing the user to check the details.

[0470] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0471] This invention relates to a system that integrates and manages structured and unstructured data, performs data analysis and prediction, and recognizes user emotions, enabling the optimization of strategies based on those emotions. The system comprises processes including data ingestion, emotion recognition, data analysis, prediction, and strategy execution.

[0472] The server first receives structured and unstructured data uploaded by users and stores them in a database. Unstructured data is converted into structured data using an analysis tool, and in the process, an emotion engine is used to recognize the emotions contained in the data. This emotion recognition involves sentiment analysis from text and voice tone analysis to extract emotional characteristics such as joy, anger, and sadness.

[0473] Next, the server uses the recognized sentiment information to perform structured data analysis. This analysis employs standard statistical analysis and machine learning, while also considering the user's sentiment data. This allows for an analysis of the impact of emotional changes on business outcomes and the creation of more precise predictions.

[0474] Predictive tools make it possible to estimate future sales and market trends based on specific emotional tendencies. Based on these detailed predictions, the server provides users with information for strategic planning.

[0475] Based on the analysis results and predictions presented by the server, users formulate individual business strategies and send them to the server as commands from their terminals. The server then implements these strategies, optimized based on sentiment information, and carries out their effective execution through execution control mechanisms.

[0476] For example, in designing an advertising campaign, it's possible to consider customer emotional tendencies and develop messages with optimized content and timing to resonate emotionally. This can improve customer engagement and maximize marketing effectiveness.

[0477] Thus, the system of the present invention supports companies in quickly and effectively utilizing data and making strategic decisions through multidimensional data processing, including emotion recognition.

[0478] The following describes the processing flow.

[0479] Step 1:

[0480] Users upload structured data (e.g., sales records in spreadsheet format) and unstructured data (e.g., customer review audio files or social media text posts) to the server. The server receives this data and stores it in data storage as structured and unstructured data, respectively.

[0481] Step 2:

[0482] The server analyzes unstructured data and converts it into structured data. Speech data is converted to text using speech recognition technology, and the emotion engine recognizes customer emotions through text analysis. Emotions are categorized as positive, negative, or neutral.

[0483] Step 3:

[0484] The server performs data analysis based on the transformed structured data. This analysis includes sales records and recognized sentiment data, and utilizes machine learning models to discover correlations and customer trends. This reveals the connection between emotions and purchasing behavior.

[0485] Step 4:

[0486] The server uses predictive tools based on data analysis results to forecast future market and sales trends. It provides users with predictions that take into account the impact of emotional changes on sales. For example, it might conclude that "sales of products with many positive reviews are likely to increase by 10% next month."

[0487] Step 5:

[0488] Users develop business strategies based on these analysis results and predictions. They then send the developed strategies to the server via their device and instruct it to begin execution. For example, a user could develop a promotional plan for a new product that leverages positive reviews.

[0489] Step 6:

[0490] The server analyzes received strategic directives and develops effective implementation plans. It communicates with external systems to manage the specific deployment of advertising campaigns and promotions. The server makes adjustments based on user-specified conditions and emotional characteristics and continuously monitors the effectiveness of the campaigns.

[0491] (Example 2)

[0492] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0493] Traditional data analysis systems only handle structured data, and therefore have the limitation of not being able to effectively utilize the emotional information contained in unstructured data. As a result, it has been difficult to make predictions and develop strategies that adequately reflect customer emotional trends, leading to a lack of competitiveness, especially in the marketing domain.

[0494] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0495] In this invention, the server includes means for receiving structured and unstructured data and storing it in data storage, means for converting unstructured data into structured data and for analyzing emotions, and means for performing data analysis based on structured data while taking emotional information into consideration. This enables predictions that reflect the user's emotions and the execution of optimized strategies based on those predictions.

[0496] "Structured data" refers to information that is organized into rows and columns, like in a database or spreadsheet, and presented in a format that is easy for machines to process.

[0497] "Unstructured data" refers to information that is not organized into a specific format, such as text documents or audio files, and is difficult for machines to analyze or process.

[0498] "Data storage" refers to a physical or virtual storage device used to permanently store received data.

[0499] "Analysis means" refers to a process or technique for converting unstructured data into structured data and extracting necessary information.

[0500] "Analysis methods for recognizing emotions" refer to technologies for extracting emotional characteristics from text or audio and interpreting them.

[0501] A "processing method" is a computational process used to analyze data and interpret the results.

[0502] A "predictive tool" is a technology used to estimate future trends and outcomes based on past data and analysis results.

[0503] A "control mechanism" is a process that directs specific actions to implement a formulated strategy based on prediction results.

[0504] "Emotional information" refers to data that indicates the psychological state of the user or object, and is extracted mechanically as part of the analysis.

[0505] An "optimized strategy" is a plan that is tailored to achieve set goals in the most efficient way, based on all available information.

[0506] In this invention, the server first receives structured and unstructured data provided by the user. This data is securely stored in data storage and protected using encryption technology. Specifically, for example, a database management system is used.

[0507] Next, the server uses a natural language processing engine and speech analysis tools to convert unstructured data into a structured format. For example, common machine learning libraries are used for natural language processing. At this stage, an emotion analysis engine operates to recognize emotions and extract emotional features such as joy, anger, and sadness from the data. For example, a natural language toolkit is used for processing text data, and a speech analysis API is used for processing speech data.

[0508] Subsequently, the server performs data analysis based on structured data and sentiment information. This analysis includes methods using machine learning models to reveal past data patterns and trends. By using common machine learning frameworks, rapid and efficient data processing is achieved. This makes it possible to quantitatively understand the impact of changes in sentiment on business outcomes.

[0509] Based on the analysis results, the server uses a generative AI model to predict future sales and market trends. This prediction helps in strategy formulation and provides users with specific and useful advice. An example of a prompt might be, "Predict customer emotional responses to this week's advertising campaign and propose an effective strategy."

[0510] Finally, users develop customized strategies on their devices based on predictive information supported by data provided by the server. These strategies are then returned to the server, optimized to incorporate emotional information, and finally implemented. As a result, companies can make strategic decisions based on customer emotions, leading to more effective marketing and enhanced customer relationship management.

[0511] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0512] Step 1:

[0513] The server receives structured and unstructured data from users and stores them in data storage. Input is data files uploaded by users, and output is data stored in storage. The received data is protected using encryption technology to ensure secure access.

[0514] Step 2:

[0515] The server uses a natural language processing engine and speech analysis tools to convert unstructured data into structured format. The input is unstructured data (e.g., text documents or audio files), and the output is analyzed structured data. This conversion process includes extracting keywords from text and audio and converting them into standardized data formats.

[0516] Step 3:

[0517] The server activates the sentiment analysis engine and recognizes sentiment information in the structured data. The input is the structured data obtained in step 2, and the output is the expanded structured data containing sentiment features. Specifically, this involves using a text analysis algorithm to extract sentiment features such as joy, anger, and sadness.

[0518] Step 4:

[0519] The server performs data analysis while considering emotional information. The input is structured data containing emotional features, and the output is the analysis results. In this step, a machine learning model is used to reveal business trends and patterns by relating past data analysis with the influence of emotions.

[0520] Step 5:

[0521] The server uses a generative AI model based on the analysis results to predict future sales and market trends. The input is the data analysis results, and the output is a prediction report. In this step, deep learning techniques are used to make more precise predictions, and the generative AI model generates prompts on how to respond.

[0522] Step 6:

[0523] Users develop business strategies on their devices based on forecast reports provided by the server. The input is the forecast report, and the output is the formulated strategic directive. In this process, users plan advertising campaigns and sales strategies based on the analyzed data.

[0524] Step 7:

[0525] The server receives strategic commands from the user, utilizes emotional information to optimize and execute the strategy. The input is the strategic command from the user, and the output is the optimized strategy that has been executed. Specific actions include adjusting the appropriate timing and content of ad delivery.

[0526] (Application Example 2)

[0527] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0528] In modern communication, information delivery that takes user emotions into consideration is extremely limited. Existing advertising and information delivery systems are primarily based on users' past behavioral history and attribute information, and there are virtually no real-time means of delivering information that takes into account the user's current emotional state. As a result, users may not receive information that is truly valuable to them. There is a need to improve this situation and realize means of delivering information that accurately captures users' emotions.

[0529] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0530] In this invention, the server includes means for receiving structured and unstructured data and storing them in data storage, means for converting unstructured data into structured data and for extracting emotional characteristics, and means for processing information based on emotional characteristics in real time. This makes it possible to analyze the user's emotional state in real time and provide optimal information accordingly.

[0531] "Structured data" refers to data that is stored in a formatted form, such as in a database, and can be easily searched and analyzed.

[0532] "Unstructured data" refers to data that does not have a specific format and exists in a wide variety of forms, such as text, images, and audio.

[0533] "Data storage" refers to a platform for storing digital data, including storage systems on servers and in the cloud.

[0534] "Analysis methods" refer to the processes and tools used to analyze unstructured data and convert it into structured data.

[0535] "Emotional features" refer to characteristics that indicate human emotional states such as joy, sadness, and anger, extracted from text or audio.

[0536] "Processing means" refers to computer programs or algorithms used to analyze data and generate specific outputs related to the user.

[0537] The system of the present invention enables real-time information provision by integrating structured and unstructured data and performing user emotion recognition based on these. The server first receives the user's structured and unstructured data and stores it in data storage. Unstructured data is converted into structured data using analysis means, and emotion features are extracted. This process uses software that implements algorithms for natural language processing and emotion analysis. Specifically, the NLP library TextBlob can be used.

[0538] After emotional characteristics are extracted, the server uses processing tools to generate information tailored to the user's emotional state, such as advertisements and related information, in real time and provide it to the user's terminal. This provides optimal information that fits the user's emotions, improving the relevance and value of the information.

[0539] As a concrete example, if a user posts "It was a fun day" on social media, the server could use TextBlob to extract positive emotions and provide promotional information about recreational facilities. This system can be implemented as a Python-based application and utilizes a generative AI model to perform accelerated sentiment analysis.

[0540] An example of a prompt could be: "Analyze the sentiment of the text and suggest an advertising message based on that sentiment."

[0541] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0542] Step 1:

[0543] The server stores structured and unstructured data received from users in data storage. Input is user text and audio data, and output is the retention of this data in storage. The stored data forms the basis for subsequent processing.

[0544] Step 2:

[0545] The server receives unstructured data and converts it into structured data using analysis tools. This process uses natural language processing libraries such as TextBlob to perform sentiment analysis on text data and extract emotional features. The input is unstructured text data, and the output is structured data including sentiment tags (e.g., positive, negative, neutral).

[0546] Step 3:

[0547] The server generates user-specific information based on sentiment characteristics extracted from processed data. Here, it retrieves advertisements and information corresponding to predefined sentiment tags from a database. The input consists of sentiment characteristics and historical advertising data, while the output is optimized advertising information. This allows for real-time selection of information tailored to the user's emotional state.

[0548] Step 4:

[0549] The device displays information received from the server on the user interface. During display, the visuals are adjusted to match the user's emotions and presented in an appealing format. The input is optimized advertising information, and the output is the display of advertisements to the user. This process allows the user to receive information that aligns with their mood.

[0550] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0551] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0552] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0553] [Fourth Embodiment]

[0554] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0555] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0556] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0557] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0558] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0559] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0560] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0561] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0562] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0563] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0564] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0565] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0566] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0567] This invention relates to a system for integrating and managing structured and unstructured data and executing truly useful business strategies through data analysis and forecasting. The system includes processes for data ingestion, analysis of unstructured data, data analysis, forecasting, and strategy execution.

[0568] The server is designed to receive both structured and unstructured data and stores this data in data storage. Unstructured data is converted into structured data using specific analytical methods. Through this analysis, the unstructured data is formatted into a format that is easily visualized and measured using techniques such as natural language processing and image recognition.

[0569] Next, the server performs data analysis using the analyzed structured data. This is a process to extract useful information and trends from past data. The insights gained through this analysis process are used to estimate future outcomes using predictive tools. For example, this can be used to forecast sales or customer behavior.

[0570] Users can formulate specific business strategies based on analysis results and predictions provided by the server. They can then send the selected strategy to the system via their terminal to initiate its execution. The server receives the command to execute the strategy and implements it effectively through its control mechanisms.

[0571] As a concrete example, consider a scenario where a user uploads unstructured audio data of customer feedback to the system. The server converts this audio data into text and uses natural language processing to analyze the customer's emotions and intentions. Based on these results, the server predicts increases or decreases in sales and the need for product improvements. The user then develops a new marketing campaign based on these predictions and sends execution commands through the system. The server manages the resources necessary to run the campaign and deploys it at the optimal time.

[0572] Thus, the system of the present invention enables companies to comprehensively utilize data and make quick and effective decisions.

[0573] The following describes the processing flow.

[0574] Step 1:

[0575] Users upload structured data (e.g., database export files) and unstructured data (e.g., audio files and text documents) to the server. The server receives these files and stores the structured data in the `structured_data` list and the unstructured data in the `unstructured_data` list.

[0576] Step 2:

[0577] The server initiates a process to analyze unstructured data and convert it into structured data. For example, after converting audio data to text, it uses natural language processing to extract keywords and perform sentiment analysis from the text. The server then adds the results of this conversion to the structured_data list as structured information.

[0578] Step 3:

[0579] The server performs data analysis based on structured data lists. This analysis includes applying statistical analysis and machine learning algorithms to gain useful insights from the data. For example, it identifies best-selling products and seasonal trends based on historical sales data.

[0580] Step 4:

[0581] The server makes future predictions based on the results of data analysis. This includes using predictive models to forecast future sales and estimate market trends. The prediction results are provided to users to help them in strategic planning.

[0582] Step 5:

[0583] The user develops a specific business strategy based on the prediction results from the server. The strategy is then sent to the server as a command using a terminal, initiating the strategy's execution process.

[0584] Step 6:

[0585] The server executes the plan based on the strategic directives it receives. This execution includes managing the optimal deployment of campaigns and promotions while coordinating with external systems. The server monitors overall progress and makes adjustments to ensure success.

[0586] (Example 1)

[0587] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0588] Traditional data management systems have faced challenges in effectively integrating structured and unstructured information and quickly reflecting it in actual business strategies. Unstructured information is diverse in format and difficult to analyze, resulting in significant time and effort required to gain practical insights. Furthermore, there has been a lack of efficient means to execute strategies based on future predictions.

[0589] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0590] In this invention, the server includes means for receiving structured and unstructured information and storing it in information memory; means for analyzing unstructured information to convert it into structured information; means for processing information based on the structured information; means for analyzing unstructured information using natural language processing or image recognition technology; and means for transmitting policy execution commands through a terminal. This enables rapid decision-making and effective implementation of strategies by comprehensively utilizing structured and unstructured information.

[0591] "Structured information" refers to information organized according to a specific format or structure, and is typically stored in databases, spreadsheets, and other similar formats.

[0592] "Unstructured information" refers to information that does not adhere to a specific format and exists in a variety of forms, including audio data, image data, and text documents.

[0593] "Information storage" refers to physical or digital storage devices for holding structured and unstructured information.

[0594] "Information analysis means" refers to technologies and tools that have the function of analyzing unstructured information and converting it into structured information.

[0595] "Processing means for performing information analysis" refers to technologies that have the function of performing calculations and models to derive trends and patterns based on structured information.

[0596] "Natural language processing" refers to the technology that enables computers to understand, interpret, and generate human language.

[0597] "Image recognition technology" refers to the technology used to analyze image data and recognize specific objects or features.

[0598] A "terminal" refers to a digital device used by a user to communicate with a server and perform operations or transmit information.

[0599] A "policy execution command" refers to an instruction that prompts a server to begin carrying out specific actions or procedures.

[0600] This invention is a system that enables companies to integrate and utilize structured and unstructured information to formulate and execute useful business strategies. Specific embodiments are described below.

[0601] The server performs its primary functions, primarily information management. First, the server receives structured and unstructured information from users and stores it in its information storage device. Structured information is retrieved from existing databases, while unstructured information includes various data formats such as audio, images, and text. This unstructured information is analyzed by the server using natural language processing and image recognition technologies and converted into structured information. This process utilizes application-specific analysis software.

[0602] The analyzed information is processed on an information analysis platform within the server. This platform uses advanced machine learning algorithms, including generative AI models, to extract customer behavior patterns and market trends from past data. Based on these results, the server predicts future outcomes and provides them to the user.

[0603] Users receive analysis results and predictions from the server via their terminals and formulate new business strategies based on them. In this process, users send instructions from their terminals to the server to prompt specific actions.

[0604] As a concrete example, when a user uploads audio data of customer feedback to a server, the server converts the data into text and performs a sentiment analysis of the customer through natural language processing. Based on the insights gained from this analysis, the user can decide on strategies for increasing sales or improving products. They then send these strategies to the server as execution commands, and the server carries out campaigns and operations according to those commands.

[0605] An example of a prompt for a generative AI model is, "Explain the process of converting audio data into text and using that content to make sales forecasts." This system supports companies in making quick and effective decisions by utilizing information from multiple perspectives.

[0606] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0607] Step 1:

[0608] Users upload structured and unstructured information to the server using their devices. Unstructured information includes customer voice data and email text. This information is stored in a data storage device by the server. Inputs are in various data formats, and these data are securely recorded as outputs.

[0609] Step 2:

[0610] The server uses natural language processing and image recognition technologies to analyze unstructured information. Specifically, it converts audio data into text and uses natural language processing tools to analyze customer emotions and intentions. The input is uploaded unstructured information, and the output is analyzed structured information.

[0611] Step 3:

[0612] The server performs information analysis based on structured information. This includes processes that utilize machine learning algorithms, such as generative AI models, to derive useful trends and patterns from past data. The input is structured information, and the output is the analyzed insights and patterns.

[0613] Step 4:

[0614] The server uses prediction tools based on the analysis results to estimate future outcomes. It performs simulations of sales trends and customer behavior, quantifying future prospects. The input is the analyzed information, and the output is the predicted future results.

[0615] Step 5:

[0616] Users receive prediction results from the server and formulate new business strategies based on the information displayed on their devices. These strategies are then reflected in specific policies, such as the planning of more proactive marketing campaigns. The input is the prediction results, and the output is the formulated strategy.

[0617] Step 6:

[0618] The user sends the formulated strategy to the server as an execution command via their terminal. The server receives this command and begins implementing the policy using its control mechanisms. The input is the command from the user, and the output is the specific actions taken to implement it.

[0619] (Application Example 1)

[0620] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] In today's business environment, it is crucial to extract useful information from diverse data and formulate and execute concrete strategies. However, in situations where structured and unstructured data coexist, efficiently analyzing them and providing customized proposals for individual customers is difficult. Therefore, companies are required to utilize data quickly and effectively to enhance their competitiveness.

[0622] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0623] In this invention, the server includes means for receiving structured and unstructured data and storing it in a data storage device, analysis means for converting unstructured data into structured data, and processing means for performing data analysis based on the structured data. This makes it possible to propose products optimized for the customer.

[0624] "Structured data" refers to data that exists in a well-organized, structured format, such as in databases or spreadsheets, making it easy to search and analyze.

[0625] "Unstructured data" refers to data in formats that do not have a specific structure, such as audio, text, and image data, and is difficult to analyze in its original form.

[0626] A "data storage device" is a physical or virtual storage device used to store received data.

[0627] "Analysis methods" refer to the process of processing data using technologies such as natural language processing and image recognition in order to convert unstructured data into structured data.

[0628] "Processing means" refers to a function that uses structured data to perform statistical analysis and machine learning-based data analysis to extract useful information.

[0629] A "predictive function" is a technology that estimates future trends and outcomes based on analyzed data and generates prediction results.

[0630] The "control function" is the process of issuing commands to execute strategies formulated based on prediction results and managing their progress.

[0631] The "suggestion function" is a system that provides optimized product information and purchase suggestions based on customer history and feedback.

[0632] A "communication function" is an interface for exchanging data with an external information processing device.

[0633] The system of the present invention integrates the functions of data reception, analysis, interpretation, prediction, proposal, and execution. The server receives structured and unstructured data and stores it in a data storage device. Unstructured data is converted into structured data using natural language processing and image recognition technologies, and this is carried out through analysis means. The technologies used can be combined with conventional database management systems and include NLP engines and image recognition software.

[0634] The server performs data analysis using the transformed structured data and makes future predictions using machine learning models based on the features extracted from the data. Machine learning frameworks such as TensorFlow and PyTorch are utilized for this purpose. For example, it can recommend the most suitable products and services for the future based on customer purchase history and feedback.

[0635] The terminal operates a generating AI model based on user instructions, formulates a strategy based on the prediction results, and sends commands to the server for execution. An example of a prompt might be: "Analyze past customer feedback and generate a list of products they are most likely to purchase next. Pay particular attention to feedback regarding sizing and product color from unstructured voice data."

[0636] As a result, this information can be used to develop new marketing campaigns and optimize pricing, enabling companies to respond flexibly and quickly to customer needs.

[0637] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0638] Step 1:

[0639] The server receives customer feedback (unstructured data) in the form of voice and text, as well as purchase history (structured data), from smartphone devices. The received data is stored in a data storage device. Here, the input is voice data and past purchase history, and the output is the raw data stored in the data storage device.

[0640] Step 2:

[0641] The server analyzes the received unstructured audio data using a natural language processing engine and converts it into text. The result is a format that allows for understanding the customer's emotions and intentions. The input is audio data, and the output is structured text data. Specifically, speech recognition technology is used to convert the audio into text, followed by text analysis to extract the user's emotions.

[0642] Step 3:

[0643] The server performs data analysis using the converted text data and purchase history. Machine learning models are used to model customer purchasing trends and preferences. The input is structured text data and purchase history, and the output is a dataset representing customer characteristics. Specifically, cluster analysis and classification algorithms are used to analyze the data.

[0644] Step 4:

[0645] The server uses a machine learning model based on data analysis results to predict future purchases. It generates specific product recommendations and formulates target products and pricing strategies. The input is customer characteristic data, and the output is a list of recommended products. The product suggestions generated by the AI ​​model are optimized by referring to prompt text.

[0646] Step 5:

[0647] The device notifies the user of the generated recommendation information and presents the most suitable products to the customer. The user can then make a purchase decision based on this information. Here, the input is the recommendation information, and the output is the notification or display to the user. Specifically, a list of recommended products is displayed on the smartphone, allowing the user to check the details.

[0648] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0649] This invention relates to a system that integrates and manages structured and unstructured data, performs data analysis and prediction, and recognizes user emotions, enabling the optimization of strategies based on those emotions. The system comprises processes including data ingestion, emotion recognition, data analysis, prediction, and strategy execution.

[0650] The server first receives structured and unstructured data uploaded by users and stores them in a database. Unstructured data is converted into structured data using an analysis tool, and in the process, an emotion engine is used to recognize the emotions contained in the data. This emotion recognition involves sentiment analysis from text and voice tone analysis to extract emotional characteristics such as joy, anger, and sadness.

[0651] Next, the server uses the recognized sentiment information to perform structured data analysis. This analysis employs standard statistical analysis and machine learning, while also considering the user's sentiment data. This allows for an analysis of the impact of emotional changes on business outcomes and the creation of more precise predictions.

[0652] Predictive tools make it possible to estimate future sales and market trends based on specific emotional tendencies. Based on these detailed predictions, the server provides users with information for strategic planning.

[0653] Based on the analysis results and predictions presented by the server, users formulate individual business strategies and send them to the server as commands from their terminals. The server then implements these strategies, optimized based on sentiment information, and carries out their effective execution through execution control mechanisms.

[0654] For example, in designing an advertising campaign, it's possible to consider customer emotional tendencies and develop messages with optimized content and timing to resonate emotionally. This can improve customer engagement and maximize marketing effectiveness.

[0655] Thus, the system of the present invention supports companies in quickly and effectively utilizing data and making strategic decisions through multidimensional data processing, including emotion recognition.

[0656] The following describes the processing flow.

[0657] Step 1:

[0658] Users upload structured data (e.g., sales records in spreadsheet format) and unstructured data (e.g., customer review audio files or social media text posts) to the server. The server receives this data and stores it in data storage as structured and unstructured data, respectively.

[0659] Step 2:

[0660] The server analyzes unstructured data and converts it into structured data. Speech data is converted to text using speech recognition technology, and the emotion engine recognizes customer emotions through text analysis. Emotions are categorized as positive, negative, or neutral.

[0661] Step 3:

[0662] The server performs data analysis based on the transformed structured data. This analysis includes sales records and recognized sentiment data, and utilizes machine learning models to discover correlations and customer trends. This reveals the connection between emotions and purchasing behavior.

[0663] Step 4:

[0664] The server uses predictive tools based on data analysis results to forecast future market and sales trends. It provides users with predictions that take into account the impact of emotional changes on sales. For example, it might conclude that "sales of products with many positive reviews are likely to increase by 10% next month."

[0665] Step 5:

[0666] Users develop business strategies based on these analysis results and predictions. They then send the developed strategies to the server via their device and instruct it to begin execution. For example, a user could develop a promotional plan for a new product that leverages positive reviews.

[0667] Step 6:

[0668] The server analyzes received strategic directives and develops effective implementation plans. It communicates with external systems to manage the specific deployment of advertising campaigns and promotions. The server makes adjustments based on user-specified conditions and emotional characteristics and continuously monitors the effectiveness of the campaigns.

[0669] (Example 2)

[0670] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0671] Traditional data analysis systems only handle structured data, and therefore have the limitation of not being able to effectively utilize the emotional information contained in unstructured data. As a result, it has been difficult to make predictions and develop strategies that adequately reflect customer emotional trends, leading to a lack of competitiveness, especially in the marketing domain.

[0672] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0673] In this invention, the server includes means for receiving structured and unstructured data and storing it in data storage, means for converting unstructured data into structured data and for analyzing emotions, and means for performing data analysis based on structured data while taking emotional information into consideration. This enables predictions that reflect the user's emotions and the execution of optimized strategies based on those predictions.

[0674] "Structured data" refers to information that is organized into rows and columns, like in a database or spreadsheet, and presented in a format that is easy for machines to process.

[0675] "Unstructured data" refers to information that is not organized into a specific format, such as text documents or audio files, and is difficult for machines to analyze or process.

[0676] "Data storage" refers to a physical or virtual storage device used to permanently store received data.

[0677] "Analysis means" refers to a process or technique for converting unstructured data into structured data and extracting necessary information.

[0678] "Analysis methods for recognizing emotions" refer to technologies for extracting emotional characteristics from text or audio and interpreting them.

[0679] A "processing method" is a computational process used to analyze data and interpret the results.

[0680] A "predictive tool" is a technology used to estimate future trends and outcomes based on past data and analysis results.

[0681] A "control mechanism" is a process that directs specific actions to implement a formulated strategy based on prediction results.

[0682] "Emotional information" refers to data that indicates the psychological state of the user or object, and is extracted mechanically as part of the analysis.

[0683] An "optimized strategy" is a plan that is tailored to achieve set goals in the most efficient way, based on all available information.

[0684] In this invention, the server first receives structured and unstructured data provided by the user. This data is securely stored in data storage and protected using encryption technology. Specifically, for example, a database management system is used.

[0685] Next, the server uses a natural language processing engine and speech analysis tools to convert unstructured data into a structured format. For example, common machine learning libraries are used for natural language processing. At this stage, an emotion analysis engine operates to recognize emotions and extract emotional features such as joy, anger, and sadness from the data. For example, a natural language toolkit is used for processing text data, and a speech analysis API is used for processing speech data.

[0686] Subsequently, the server performs data analysis based on structured data and sentiment information. This analysis includes methods using machine learning models to reveal past data patterns and trends. By using common machine learning frameworks, rapid and efficient data processing is achieved. This makes it possible to quantitatively understand the impact of changes in sentiment on business outcomes.

[0687] Based on the analysis results, the server uses a generative AI model to predict future sales and market trends. This prediction helps in strategy formulation and provides users with specific and useful advice. An example of a prompt might be, "Predict customer emotional responses to this week's advertising campaign and propose an effective strategy."

[0688] Finally, users develop customized strategies on their devices based on predictive information supported by data provided by the server. These strategies are then returned to the server, optimized to incorporate emotional information, and finally implemented. As a result, companies can make strategic decisions based on customer emotions, leading to more effective marketing and enhanced customer relationship management.

[0689] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0690] Step 1:

[0691] The server receives structured and unstructured data from users and stores them in data storage. Input is data files uploaded by users, and output is data stored in storage. The received data is protected using encryption technology to ensure secure access.

[0692] Step 2:

[0693] The server uses a natural language processing engine and speech analysis tools to convert unstructured data into structured format. The input is unstructured data (e.g., text documents or audio files), and the output is analyzed structured data. This conversion process includes extracting keywords from text and audio and converting them into standardized data formats.

[0694] Step 3:

[0695] The server activates the sentiment analysis engine and recognizes sentiment information in the structured data. The input is the structured data obtained in step 2, and the output is the expanded structured data containing sentiment features. Specifically, this involves using a text analysis algorithm to extract sentiment features such as joy, anger, and sadness.

[0696] Step 4:

[0697] The server performs data analysis while considering emotional information. The input is structured data containing emotional features, and the output is the analysis results. In this step, a machine learning model is used to reveal business trends and patterns by relating past data analysis with the influence of emotions.

[0698] Step 5:

[0699] The server uses a generative AI model based on the analysis results to predict future sales and market trends. The input is the data analysis results, and the output is a prediction report. In this step, deep learning techniques are used to make more precise predictions, and the generative AI model generates prompts on how to respond.

[0700] Step 6:

[0701] Users develop business strategies on their devices based on forecast reports provided by the server. The input is the forecast report, and the output is the formulated strategic directive. In this process, users plan advertising campaigns and sales strategies based on the analyzed data.

[0702] Step 7:

[0703] The server receives strategic commands from the user, utilizes emotional information to optimize and execute the strategy. The input is the strategic command from the user, and the output is the optimized strategy that has been executed. Specific actions include adjusting the appropriate timing and content of ad delivery.

[0704] (Application Example 2)

[0705] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0706] In modern communication, information delivery that takes user emotions into consideration is extremely limited. Existing advertising and information delivery systems are primarily based on users' past behavioral history and attribute information, and there are virtually no real-time means of delivering information that takes into account the user's current emotional state. As a result, users may not receive information that is truly valuable to them. There is a need to improve this situation and realize means of delivering information that accurately captures users' emotions.

[0707] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0708] In this invention, the server includes means for receiving structured and unstructured data and storing them in data storage, means for converting unstructured data into structured data and for extracting emotional characteristics, and means for processing information based on emotional characteristics in real time. This makes it possible to analyze the user's emotional state in real time and provide optimal information accordingly.

[0709] "Structured data" refers to data that is stored in a formatted form, such as in a database, and can be easily searched and analyzed.

[0710] "Unstructured data" refers to data that does not have a specific format and exists in a wide variety of forms, such as text, images, and audio.

[0711] "Data storage" refers to a platform for storing digital data, including storage systems on servers and in the cloud.

[0712] "Analysis methods" refer to the processes and tools used to analyze unstructured data and convert it into structured data.

[0713] "Emotional features" refer to characteristics that indicate human emotional states such as joy, sadness, and anger, extracted from text or audio.

[0714] "Processing means" refers to computer programs or algorithms used to analyze data and generate specific outputs related to the user.

[0715] The system of the present invention enables real-time information provision by integrating structured and unstructured data and performing user emotion recognition based on these. The server first receives the user's structured and unstructured data and stores it in data storage. Unstructured data is converted into structured data using analysis means, and emotion features are extracted. This process uses software that implements algorithms for natural language processing and emotion analysis. Specifically, the NLP library TextBlob can be used.

[0716] After emotional characteristics are extracted, the server uses processing tools to generate information tailored to the user's emotional state, such as advertisements and related information, in real time and provide it to the user's terminal. This provides optimal information that fits the user's emotions, improving the relevance and value of the information.

[0717] As a concrete example, if a user posts "It was a fun day" on social media, the server could use TextBlob to extract positive emotions and provide promotional information about recreational facilities. This system can be implemented as a Python-based application and utilizes a generative AI model to perform accelerated sentiment analysis.

[0718] An example of a prompt could be: "Analyze the sentiment of the text and suggest an advertising message based on that sentiment."

[0719] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0720] Step 1:

[0721] The server stores structured and unstructured data received from users in data storage. Input is user text and audio data, and output is the retention of this data in storage. The stored data forms the basis for subsequent processing.

[0722] Step 2:

[0723] The server receives unstructured data and converts it into structured data using analysis tools. This process uses natural language processing libraries such as TextBlob to perform sentiment analysis on text data and extract emotional features. The input is unstructured text data, and the output is structured data including sentiment tags (e.g., positive, negative, neutral).

[0724] Step 3:

[0725] The server generates user-specific information based on sentiment characteristics extracted from processed data. Here, it retrieves advertisements and information corresponding to predefined sentiment tags from a database. The input consists of sentiment characteristics and historical advertising data, while the output is optimized advertising information. This allows for real-time selection of information tailored to the user's emotional state.

[0726] Step 4:

[0727] The device displays information received from the server on the user interface. During display, the visuals are adjusted to match the user's emotions and presented in an appealing format. The input is optimized advertising information, and the output is the display of advertisements to the user. This process allows the user to receive information that aligns with their mood.

[0728] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0729] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0730] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0731] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0732] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0733] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0734] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0735] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0736] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0737] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0738] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0739] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0740] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0741] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0742] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0743] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0744] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0745] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0746] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0747] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0748] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0749] The following is further disclosed regarding the embodiments described above.

[0750] (Claim 1)

[0751] A means for receiving structured and unstructured data and storing them in data storage,

[0752] An analysis method for converting unstructured data into structured data,

[0753] Processing methods for performing data analysis based on structured data,

[0754] A prediction method that performs predictions based on the analysis results,

[0755] Control measures for formulating and executing forecast-based strategies,

[0756] A system that includes this.

[0757] (Claim 2)

[0758] The system according to claim 1, comprising means for using machine learning models in data analysis.

[0759] (Claim 3)

[0760] The system according to claim 1, further comprising communication means for issuing strategic execution commands in cooperation with an external system.

[0761] "Example 1"

[0762] (Claim 1)

[0763] A means for receiving structured and unstructured information and storing them in information memory,

[0764] Information analysis means for converting unstructured information into structured information,

[0765] A processing method for performing information analysis based on structured information,

[0766] A predictive means for estimating future results based on the analysis results,

[0767] Control measures for formulating and implementing policies based on predictions,

[0768] A means for analyzing unstructured information using natural language processing or image recognition technology,

[0769] A means of transmitting policy execution commands through a terminal,

[0770] A system that includes this.

[0771] (Claim 2)

[0772] The system according to claim 1, comprising means for using a generative AI model in information analysis.

[0773] (Claim 3)

[0774] The system according to claim 1, further comprising communication means for issuing policy execution commands in cooperation with an external system.

[0775] "Application Example 1"

[0776] (Claim 1)

[0777] A means for receiving structured and unstructured data and storing them in a data storage device,

[0778] An analysis method for converting unstructured data into structured data,

[0779] Processing methods for performing data analysis based on structured data,

[0780] A function that predicts the future based on the analysis results,

[0781] Control functions for formulating and executing forecast-based strategies,

[0782] A suggestion function to provide products optimized for the customer,

[0783] A system that includes this.

[0784] (Claim 2)

[0785] The system according to claim 1, comprising means for using machine learning models in data analysis and proposals.

[0786] (Claim 3)

[0787] The system according to claim 1, further comprising a communication function for issuing strategic execution commands in cooperation with an external information processing device.

[0788] "Example 2 of combining an emotion engine"

[0789] (Claim 1)

[0790] A means for receiving structured and unstructured data and storing them in data storage,

[0791] An analytical method for converting unstructured data into structured data and recognizing emotions,

[0792] A processing method for performing data analysis based on structured data while taking emotional information into consideration,

[0793] A prediction method that performs predictions based on analysis results and sentiment information,

[0794] A control mechanism for formulating prediction-based strategies and executing optimized strategies using emotional information,

[0795] A system that includes this.

[0796] (Claim 2)

[0797] The system according to claim 1, comprising means for using machine learning models in data analysis.

[0798] (Claim 3)

[0799] The system according to claim 1, further comprising communication means for issuing strategic execution commands in cooperation with an external system.

[0800] "Application example 2 when combining with an emotional engine"

[0801] 2. Extraction of novel technical aspects of application examples

[0802] The key feature of this application is the "display of advertisements based on sentiment analysis." This aspect is considered a novel technology.

[0803] 3. Conception of a new invention

[0804] Based on sentiment analysis, we envision a system that provides real-time information tailored to the user's emotional state based on structured data.

[0805] 4. Rewriting of Claims

[0806] Rewrite it as follows:

[0807] (Claim 1)

[0808] A means for receiving structured and unstructured data and storing them in data storage,

[0809] A means for converting unstructured data into structured data and for extracting emotional features,

[0810] A processing method for providing information based on emotional characteristics in real time,

[0811] A control means for making predictions using the results of sentiment analysis and formulating strategies based on those predictions,

[0812] A system that includes this.

[0813] (Claim 2)

[0814] The system according to claim 1, which includes means for using machine learning models in data analysis and provides information according to the results of sentiment analysis.

[0815] (Claim 3)

[0816] The system according to claim 1, further comprising communication means for issuing emotion-based information provision commands in cooperation with an external information provision device. [Explanation of Symbols]

[0817] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving structured and unstructured data and storing them in data storage, An analysis method for converting unstructured data into structured data, Processing methods for performing data analysis based on structured data, A prediction method that performs predictions based on the analysis results, Control measures for formulating and executing forecast-based strategies, A system that includes this.

2. The system according to claim 1, comprising means for using a machine learning model in data analysis.

3. The system according to claim 1, further comprising communication means for issuing strategic execution commands in cooperation with an external system.

Images