system

Abstract

Provide a system. A means for collecting data through APIs of different platforms; A means of organizing and consolidating the collected data, A means to analyze data using machine learning models and personalize it according to user needs; A system including means for providing the personalized data to a user.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, the proliferation of diverse devices and cloud services has led to the dispersion of data generated, making it difficult for individuals and companies to centrally manage and efficiently utilize this data. This situation can lead to problems such as users being unable to quickly access the information they need, which can result in reduced productivity. In particular, the increased effort required to integrate and manage data across different platforms places an increased burden on users. This can result in a significant waste of time and effort, as well as the risk of missing or overlooking information. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system that includes a means for collecting data through APIs of different platforms, a means for organizing and integrating the collected data, a means for analyzing the data using a machine learning model and personalizing it according to user needs, and a means for providing the personalized data to the user. This system enables centralized management of data from different platforms and efficient use of the data. Specifically, by providing a means for merging collected data, eliminating duplicates, and generating an integrated data structure, and a means for extracting and presenting high-priority information based on user requests, the system allows users to quickly access the information they need, improving productivity.

[0006] "API" stands for Application Programming Interface, a standardized interface for exchanging information between different software programs.

[0007] "Means of collecting data" refers to the functions and processes used to obtain and collect data through APIs of different platforms.

[0008] "Data organization and integration means" refers to the methods and functions used to structure collected data, remove duplicates and unnecessary information, and turn it into a single, integrated data set.

[0009] A "machine learning model" is an algorithm used to learn from large amounts of data to automate specific tasks.

[0010] "Means of analyzing data" refers to the functions and processes used to analyze collected, organized, and integrated data and find patterns and relationships.

[0011] "Means for personalization according to user needs" refers to the functions and processes for customizing data based on analyzed data to suit the individual needs and preferences of users.

[0012] "Means of providing to the user" refers to the methods and functions for notifying or presenting personalized data to the user in a format that is easily accessible to the user.

[0013] "Data merging means" refers to methods or functions for combining multiple different data sets into a single coherent data set.

[0014] "Deduplication measures" refers to the functions and processes used to identify and remove duplicate data within a collected data set.

[0015] A "data structure" is an organized format for effectively storing, managing, and using data.

[0016] "Means for extracting high-priority information based on user requests" refers to methods or functions for selecting and displaying information that is considered particularly important based on user instructions or predicted needs.

[0017] A "system" refers to an integrated setup of multiple components working together to perform a specific task. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention relates to a data integration management system that collects, organizes, and consolidates data through APIs from different platforms, and further analyzes the data using machine learning models to provide personalized information according to user needs.

[0040] System Configuration

[0041] The system mainly consists of the following components:

[0042] 1. Data collection server: Acquires data from different platforms via API.

[0043] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[0044] 3. Analysis Server: Based on the integrated data, it uses machine learning models to analyze the data and generate personalized information based on the user's needs.

[0045] 4. User Interface: The interface through which the user accesses personalized information.

[0046] Program processing

[0047] Data collection

[0048] The server connects to the APIs of the various platforms used by the user and collects data. For example, if a user uses cloud storage, a calendar app, or an email service, the server collects data from each API.

[0049] Data organization and integration

[0050] The server then organizes and consolidates the data collected from each platform, automatically eliminating duplicate data and unnecessary information. For example, calendar events and email content are consolidated and compiled into a single data set for each user.

[0051] Data Analysis and Personalization

[0052] The server then uses machine learning models to analyze the integrated data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set. For example, a businessperson might prioritize schedule changes and important emails.

[0053] Providing data to users

[0054] The device provides the analyzed personalized data to the user, and helps the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing the user to check important appointments and messages.

[0055] Specific examples

[0056] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0057] 2. The device sends requests to the server and collects data from different platforms.

[0058] 3. The server collects data from cloud storage, calendar, and email services through APIs.

[0059] 4. The server organizes and consolidates the collected data to create a coherent data set.

[0060] 5. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs.

[0061] 6. The device provides personalized information to the user through a user interface.

[0062] 7. Users can view schedules and important emails generated on their smartphone screens and take action to improve their productivity.

[0063] As described above, the system of the present invention efficiently integrates data from various devices and cloud services, providing an environment in which users can quickly access the information they need. This system is expected to significantly reduce the data management burden on users, especially business people.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[0067] Step 2:

[0068] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[0069] Step 3:

[0070] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services) to collect data. The server sends an HTTP request to the service's API endpoint and stores the collected data in memory.

[0071] Step 4:

[0072] The server organizes and integrates the collected data, automatically eliminating duplicate data and unnecessary information. For example, it aligns file information retrieved from cloud storage with calendar appointment information to create a single integrated data set.

[0073] Step 5:

[0074] The server then uses machine learning models to analyze the combined data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set.

[0075] Step 6:

[0076] The server sends the parsed personalized data to the device, formats the data in the specified format (e.g., JSON), and sends it to the device.

[0077] Step 7:

[0078] The device displays the received data through a user interface. The device displays personalized information (e.g., the latest schedule, important email notifications) on the application screen.

[0079] Step 8:

[0080] The user can review the provided data and take necessary actions. For example, the user can review the displayed schedule details, prepare for a meeting, or instantly reply to an important email.

[0081] Example 1

[0082] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0083] Conventional data collection and integration systems did not efficiently collect and organize data from different platforms, and duplicate and unnecessary data was not adequately eliminated. Furthermore, they lacked personalized functionality to enable users to quickly obtain the information they needed, reducing the efficiency of data management and information retrieval. This increased the burden of information management on users and led to problems with insufficient data utilization.

[0084] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0085] In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and personalizing it according to the user's needs; means for the terminal to send a data integration request to the server based on the user's operation; means for eliminating unnecessary and redundant information from the integrated data; and means for sending the generated personalized data to the terminal and displaying it on a user interface. This allows data from different platforms to be efficiently collected and organized, and redundant and unnecessary data to be eliminated, enabling users to quickly obtain the information they need. Furthermore, the personalization function reduces the user's burden of information management and improves data utilization.

[0086] "Different platforms" refers to different types of services and applications, such as cloud storage, calendar applications, and email services.

[0087] "API" stands for Application Programming Interface, an interface that allows software to communicate with other software.

[0088] "Means of collecting data" refers to the ability to obtain data through APIs of different platforms.

[0089] "Means for organizing and integrating data" refers to the function of organizing collected data, eliminating duplicate and unnecessary data, and generating a unified data set.

[0090] A "machine learning model" is an algorithm or model that analyzes data and personalizes information based on user behavior and needs.

[0091] "Means for personalization" refers to the function of customizing and providing information according to the needs and preferences of individual users.

[0092] The "means for the terminal to send a data integration request to the server based on a user's operation" is a function in which the user performs an operation through the device and sends a data integration request to the server based on that operation.

[0093] "Means for eliminating unnecessary and redundant information from integrated data" refers to a function that organizes and integrates collected data and eliminates redundant and unnecessary information.

[0094] "Means for transmitting the generated personalized data to the terminal and displaying it on the user interface" refers to a function for transmitting personalized information generated based on the analyzed data to the terminal and displaying it on the interface used by the user.

[0095] A "generative AI model" is an algorithm or model that uses artificial intelligence to perform generative tasks, particularly in data analysis and personalization.

[0096] A "prompt" is a natural language sentence input to a generative AI model to specify what the model should do.

[0097] The present invention relates to a data integration management system that collects data through APIs of different platforms and organizes and consolidates them. The system further analyzes the data using machine learning models and provides personalized information according to the user's needs. Specific embodiments of the present invention are described below.

[0098] System Configuration

[0099] The system mainly consists of the following components:

[0100] 1. Data collection server: Acquires data from different platforms via APIs, such as Google Calendar API, Gmail API, and cloud storage API.

[0101] 2. Data integration server: Organizes and integrates collected data, eliminating duplicate and unnecessary data.

[0102] 3. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models (e.g., TENSORFLOW (registered trademark), Scikit-learn) to generate personalized information based on the user's needs.

[0103] 4. User Interface: The interface through which users access personalized information. Examples include smartphone applications and web interfaces.

[0104] Program processing

[0105] Data collection

[0106] The server connects to the APIs of the various platforms used by the user and collects data. For example, if the user uses cloud storage, a calendar app, or an email service, the server collects data from each API (e.g., Google Calendar API, Gmail API, Dropbox API).

[0107] Data organization and integration

[0108] The server organizes and consolidates the data collected from each platform. During this process, duplicate data and unnecessary information are automatically eliminated. For example, if the same email exists on multiple devices, it will be consolidated into one. Duplicate calendar events will also be consolidated.

[0109] Data Analysis and Personalization

[0110] The server analyzes the integrated data using machine learning models (e.g., TensorFlow and Scikit-learn). The goal of the analysis is to understand the user's behavioral patterns and needs and personalize the data accordingly. For example, it can prioritize the most important meetings and tasks in a business person's schedule.

[0111] Providing data to users

[0112] The server then sends the generated personalized data to the device and provides it to the user through a user interface. For example, a notification may be displayed in a smartphone app, allowing the user to check it. Based on the user's request, the server also uses the generative AI model to extract and provide information related to the prompt sentence.

[0113] Specific examples

[0114] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0115] 2. The device sends requests to the server and collects data from different platforms.

[0116] 3. The server connects to the Google Calendar API, Gmail API, and Dropbox API to retrieve data from each service.

[0117] 4. The server analyzes the data and deletes and organizes duplicates. If there are emails with the same subject but different text, they are merged into one.

[0118] 5. The server uses machine learning models to analyze the data, extract key information relevant to the user's needs, and generate a personalized dataset.

[0119] 6. The device receives the data sent from the server and provides it to the user through a user interface. For example, important schedule or email notifications are displayed as app icons or on the screen.

[0120] 7. The user checks their schedule and important emails on their smartphone screen and takes appropriate action.

[0121] Prompt Sentence Examples

[0122] For example, by entering "What is my schedule for this week?", personalized schedule information will be displayed. By utilizing this prompt, users can obtain the information they need in natural language.

[0123] As a result, the present invention provides a system that efficiently collects and organizes data from different platforms and provides important information personalized based on user needs, thereby reducing the burden of information management on users and improving data utilization.

[0124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0125] Step 1:

[0126] The user launches the app on their smartphone and taps the "Integrate Data" button.

[0127] Input: User action (tapping the "Integrate data" button)

[0128] Output: A data integration request is sent to the server

[0129] Step 2:

[0130] The terminal transmits a data collection request to the server based on the user's operation.

[0131] Input: Requests made by the user

[0132] Output: A data collection request is sent to the server

[0133] Step 3:

[0134] Based on the received request, the server accesses the APIs of various platforms to collect data, such as the Google Calendar API, Gmail API, and cloud storage API.

[0135] Input: Data Collection Request

[0136] Output: Data collected from each platform (e.g., calendar events, emails, files in storage)

[0137] Step 4:

[0138] The server then organizes and consolidates the collected data, a process that converts data from different formats into a common format and eliminates duplicate data and unnecessary information.

[0139] Input: Data collected from each platform

[0140] Output: Cleaned and consolidated dataset

[0141] Step 5:

[0142] The server inputs the integrated data into machine learning models (e.g., TensorFlow, Scikit-learn) and analyzes the data based on user behavior patterns and needs.

[0143] Input: Cleaned and consolidated dataset

[0144] Output: Analysis results (personalized information according to user needs)

[0145] Step 6:

[0146] The server uses the analysis to generate a personalized data set, including important emails for the user and reminders for upcoming meetings.

[0147] Input: Analysis results

[0148] Output: A personalized dataset

[0149] Step 7:

[0150] The server transmits the generated personalized data to the terminal and instructs the terminal to display the data on the user interface.

[0151] Input: personalized dataset

[0152] Output: Personalized information displayed in the user interface (e.g. smartphone notifications)

[0153] Step 8:

[0154] The user can then take the next action based on the personalized information displayed on the device, such as replying to an important email or preparing to attend a scheduled meeting.

[0155] Input: Personalization information displayed in the user interface

[0156] Output: User's next action (e.g., replying to an email, preparing for a meeting)

[0157] This is the specific processing flow of this system. This system enables users to manage information efficiently, greatly improving the utilization of data.

[0158] (Application example 1)

[0159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] In modern society, it is extremely important to collect data from multiple platforms and devices and manage it centrally. However, integrating and organizing data from different platforms is not easy. Furthermore, advanced data analysis and personalization technologies are required to quickly provide users with the information they need from a wide range of information. Furthermore, it is also important to present personalized information in a visually easy-to-understand manner. The present invention has been proposed to solve these problems.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0162] In this invention, the server includes means for collecting data through APIs of different platforms, means for organizing and integrating the collected data, and means for analyzing the data using a machine learning model and personalizing it according to the user's needs. This allows the data collected from different platforms to be organized and integrated, and the information required by the user to be personalized through analysis by the machine learning model, and to be visually provided via a display device.

[0163] "Distinct platforms" refers to multiple software and service offerings, each operated by a different vendor and using a different technology stack.

[0164] "API" stands for Application Program Interface, a set of rules and tools for exchanging data and functions between software programs.

[0165] "Means of collecting data" refers to the technology and devices used to obtain information from different platforms via APIs and aggregate it centrally.

[0166] "Data organization and integration methods" are techniques and processes that organize collected data in a consistent and coherent manner, eliminating duplicate and unnecessary information.

[0167] A "machine learning model" is a computer program that uses algorithms and statistical methods to analyze data and make predictions or recognize patterns.

[0168] "Personalization tools" are technologies and methods that customize collected and integrated data to meet the specific needs of users.

[0169] A "display device" is a device for presenting visual information to a user, such as smart glasses, a tablet, or a smartphone.

[0170] A "visual display" is a screen or projector for visually displaying digital information.

[0171] The "means of providing to the user" refers to the technology and protocols that appropriately transmit and present analyzed and personalized information to the device used by the user.

[0172] An embodiment of the present invention will be described.

[0173] The system of the present invention collects data through APIs of different platforms, organizes and integrates the collected data, analyzes it using machine learning models, and provides personalized information based on user needs using a visual display. The main components include a data collection server, a data integration server, an analysis server, and a display device such as smart glasses.

[0174] First, the server connects to the APIs of various platforms to collect user activity data, purchase history, etc. The collected data is sent to a data integration server, where it is organized and de-duplicated. The organized data is then sent to an analysis server, where it is analyzed using machine learning models. The analysis results are presented to the user as personalized information optimized for them. This information is presented via a visual display such as smart glasses.

[0175] Hardware and software used

[0176] Hardware:

[0177] Smart glasses (e.g., Google Glass (registered trademark), Vuzix Blade)

[0178] Data Collection Server

[0179] Data Integration Server

[0180] Analysis Server

[0181] Customer Relationship Management System (CRM)

[0182] software:

[0183] API connection libraries (e.g., Postman, Retrofit)

[0184] Machine learning libraries (e.g. TensorFlow, scikit-learn)

[0185] Database (e.g., MySQL (registered trademark), MongoDB)

[0186] UI development frameworks (e.g., Angular, React)

[0187] Processing flow and example

[0188] First, when a user visits the store, the server collects data via APIs from different platforms, such as cloud storage, calendar services, and email services. For example, information about the user's past purchase history and browsing history is obtained. The collected data is then organized and integrated by a data integration server. During this process, duplicate and unnecessary data is eliminated, creating a consistent data set.

[0189] The analytics server then uses machine learning models based on this integrated data to analyze the information needed to meet the user's needs and generate personalized information, such as recommendations for new camping equipment based on the user's previous purchases, or new product recommendations.

[0190] Finally, the generated personalized information is provided to the user through a visual display such as smart glasses, allowing store clerks to make more accurate product recommendations to customers, which is expected to improve customer satisfaction.

[0191] Specific prompt examples

[0192] Provide information to the machine learning model using prompt statements such as:

[0193] Prompt statement:

[0194] "Please recommend some products to a male customer in his 30s who enjoys the outdoors and has purchased a lot of camping equipment in the past year."

[0195] Example response:

[0196] "I would like to recommend a new camping stove and waterproof tent to this customer. I can also offer a great deal on our recently arrived multi-purpose knife."

[0197] In this way, the system of the present invention can efficiently collect and integrate data from different platforms, use machine learning models to personalize information based on the user's needs, and provide it through a visual display, thereby improving the user experience.

[0198] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0199] Step 1:

[0200] The server connects to the APIs of different platforms (e.g., cloud storage, calendar services, email services) to collect user activity data, purchase history, etc. Specifically, it sends an HTTP request to each API endpoint and retrieves the returned data. The input to this step is the API endpoint of each platform and user authentication information, and the output is raw data of the user's activity data and purchase history.

[0201] Step 2:

[0202] The server sends the collected data to the data integration server, which organizes and integrates the data. Specifically, it standardizes the data format and eliminates duplicate data and unnecessary information. This process involves merging, for example, duplicate purchase histories or identical event information into one. The input to this step is the raw data collected from each platform, and the output is an organized and integrated dataset.

[0203] Step 3:

[0204] The server sends the consolidated data to the analysis server, where it is analyzed using a machine learning model. Specifically, data analysis is performed to predict the user's interests and preferences based on past purchase data and activity history. This process recommends products and services that best suit the user's needs. The input to this step is the consolidated data set, and the output is personalized recommendations for the user.

[0205] Step 4:

[0206] The server then transmits the analyzed personalized information to a display device (e.g., smart glasses) to provide it to the user. Specifically, the server converts the generated recommendation information into a format suitable for display on the device screen and transmits it to the smart glasses. The input of this step is the personalized information generated by the machine learning model, and the output is the recommendation information displayed on the visual display.

[0207] Step 5:

[0208] The user checks the personalized information presented through the smart glasses and takes appropriate action. Specifically, the system explains recommended products and services to the customer and helps promote purchases. The input of this step is the recommended information displayed on the smart glasses, and the output is the user's action (e.g., product suggestions or purchasing procedures).

[0209] Through each of the above steps, data collected from different platforms is efficiently integrated, and personalized information is provided to users based on the results of analysis using machine learning models, thereby achieving advanced customer service and improved service.

[0210] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0211] This invention relates to a data integration management system that collects, organizes, and integrates data through APIs from different platforms. Furthermore, it analyzes the data using machine learning models and provides personalized information according to the user's needs and emotions. By combining it with an emotion engine that recognizes the user's emotions, even more advanced personalization becomes possible.

[0212] System Configuration

[0213] The system mainly consists of the following components:

[0214] 1. Data collection server: Acquires data from different platforms via API.

[0215] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[0216] 3. Emotion engine: Recognizes user emotions and reflects them in data analysis.

[0217] 4. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models to generate personalized information based on the user's needs and emotions.

[0218] 5. User Interface: The interface through which the user accesses personalized information.

[0219] Program processing

[0220] Data collection

[0221] The server connects to the APIs of the various platforms the user uses and collects data. For example, if the user uses cloud storage, a calendar app, an email service, or a social networking service, the server collects data from each API.

[0222] Data organization and integration

[0223] The server organizes and integrates the data collected from each platform. In doing so, it automatically eliminates duplicate data and unnecessary information. For example, it aligns posts obtained from social media with communications from email services and provides them as an integrated data set.

[0224] emotion recognition

[0225] The emotion engine recognizes the user's emotions. It analyzes user input, voice data, facial expression data, etc. to identify the emotion the user is currently feeling. This emotion data is reflected in subsequent data analysis and personalization.

[0226] Data Analysis and Personalization

[0227] The server then analyzes the integrated data using machine learning models to extract the most relevant information based on the user's past behavior, current situation, and perceived emotions, generating a personalized data set. For example, if the user is feeling stressed, it will prioritize relaxation information and entertainment content.

[0228] Providing data to users

[0229] The device provides the analyzed personalized data to the user, helping the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing users to check important schedules, messages, and emotional content.

[0230] Specific examples

[0231] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0232] 2. The device sends a request to the server to collect data from different platforms.

[0233] 3. The server collects data from cloud storage, calendars, email services, and social media.

[0234] 4. The server consolidates and organizes the collected data to create a consistent data set.

[0235] 5. The emotion engine recognizes the user's emotions and provides the data to the server.

[0236] 6. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs and perceived emotions.

[0237] 7. The device provides personalized information to the user through a user interface.

[0238] 8. Users can view schedules, important emails, and emotional suggestions (such as entertainment content or relaxation methods) generated on their smartphone screen, and take action to improve productivity.

[0239] As described above, the system of the present invention efficiently integrates data from a variety of devices and cloud services, providing an environment in which users can quickly access the information they need. In particular, by combining it with an emotion engine, advanced personalization based on the user's emotional state can be achieved, making it possible to better meet individual needs. This system is expected to significantly reduce the data management burden on users, particularly business people, and improve the quality of their daily lives.

[0240] The processing flow will be explained below.

[0241] Step 1:

[0242] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[0243] Step 2:

[0244] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[0245] Step 3:

[0246] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services, SNS) to collect data. The server sends an HTTP request to the service's API endpoint and temporarily stores the collected data in memory.

[0247] Step 4:

[0248] The server organizes and integrates the data it collects. For example, it combines file data collected from cloud storage and schedule data collected from a calendar app into a single integrated data set. It also eliminates duplicate and unnecessary data.

[0249] Step 5:

[0250] The emotion engine recognizes the user's emotions. The user's voice data, text data, or image data obtained from the terminal or external device is sent to the server, which inputs it into the emotion engine. The emotion engine analyzes this data and recognizes the user's emotional state (e.g., joy, sadness, stress).

[0251] Step 6:

[0252] The server uses machine learning models to analyze the integrated data. The server analyzes the data based on the user's past behavioral history, current situation, and recognized emotions, and extracts the information most appropriate for the user. For example, if the user is feeling stressed, it will prioritize information related to relaxation.

[0253] Step 7:

[0254] The server sends the parsed personalized data to the device. The server formats the data in the specified format (e.g., JSON) and sends it to the device. The data includes important schedules, emails, SNS notifications, etc.

[0255] Step 8:

[0256] The device displays the received data through a user interface, and the device displays personalized information (e.g., the latest schedule, important email notifications, and emotional entertainment content) on the application screen.

[0257] Step 9:

[0258] The user can review the provided data and take necessary actions. For example, the user can check the displayed schedule details and prepare for a meeting. The user can also reduce stress by viewing relaxing content tailored to the user's emotions.

[0259] As described above, this system collects and integrates data from the user's various devices and cloud services, and by using an emotion engine, achieves advanced personalization according to the user's emotional state, allowing users to quickly access the information they need most and supporting efficient daily life and work performance.

[0260] Example 2

[0261] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0262] Currently, collecting, integrating, and analyzing data from different sources is a time-consuming and labor-intensive task. It is also difficult to provide personalized information that takes into account the user's emotions, making it difficult to provide appropriate services that meet the user's needs. Furthermore, issues such as data duplication and organization are not taken into consideration, resulting in a lack of user convenience.

[0263] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data through application programming interfaces of different information sources, means for organizing and integrating the collected data, means for eliminating duplicate data, means for using an emotion engine to recognize the user's emotions, means for analyzing the data using a machine learning algorithm and personalizing it according to the user's needs and emotions, and means for providing the personalized data to the user. This makes it possible to efficiently collect and integrate data from multiple information sources and provide personalized information that takes the user's emotions into consideration.

[0264] "Sources" refer to the systems and platforms that provide data, including cloud storage, calendar apps, email services, and social networking services.

[0265] An "application programming interface" is a defined set of communication rules that allow software applications to interact with other applications and services.

[0266] "Data collection" refers to the process of obtaining data from different sources and storing it on a server.

[0267] Data consolidation is the process of organizing collected data into a consistent format, linking it where necessary, and eliminating duplicate data.

[0268] "Duplicate data elimination" is the process of detecting and removing duplicate data to prevent the same information from being collected multiple times.

[0269] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions, text, voice, etc. to recognize emotions.

[0270] A "machine learning algorithm" is a mathematical model or method for learning patterns from large amounts of data and making predictions or classifications.

[0271] "Personalization" is the process of providing information optimized to the needs and emotions of individual users based on collected data and analysis results.

[0272] "Means for providing to the user" refers to an interface or notification system for providing the analyzed information in a form that the user can use.

[0273] This invention is a system that collects data through application programming interfaces (APIs) from different information sources, organizes, integrates, and analyzes them, and provides personalized information according to the user's needs and emotions. The main components of the system include the following:

[0274] 1. Data collection server: The server connects to the APIs of cloud storage, calendar apps, email services, social networking sites, etc. used by users to collect data. Specifically, it uses APIs such as Google Drive, Google Calendar, Gmail, and Twitter.

[0275] 2. Data integration server: The server integrates the collected data and organizes it into a consistent format. This process also eliminates duplicate data. Specifically, the server processes and formats the data using the Python libraries Pandas and NumPy.

[0276] 3. Emotion Engine: The emotion engine is designed to recognize the user's emotions. The server uses an emotion recognition API (e.g., Emotion API) to analyze the text data, voice data, and facial expression data entered by the user.

[0277] 4. Analytics Server: The server analyzes the data using machine learning algorithms to generate personalized information based on the user's past behavior, current situation, and perceived emotions. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch.

[0278] 5. User Interface: The device is responsible for providing personalized information to the user. It displays the information as notifications and messages through smartphone applications and web interfaces, making it easily accessible to the user.

[0279] For example, when a user launches a dedicated smartphone app and taps the "Integrate Data" button, the device sends a data collection request to the server. The server accesses various APIs to collect, organize, and integrate the necessary data. The emotion engine then analyzes the text and facial expression data to identify the user's emotions. The data is analyzed using machine learning algorithms to generate personalized information, which the device then provides to the user as notifications or messages. This series of processes allows users to quickly access the information they need and reduces the burden of information management.

[0280] An example prompt is:

[0281] "I want to develop a system that integrates and organizes data from different platforms, recognizes user emotions, and provides analytical data. I will use EmotionAPI for emotion recognition and TensorFlow for machine learning. Please explain the program process for this system step by step."

[0282] This system is expected to improve the quality of daily life and business by enabling users to integrate data from a variety of sources and obtain the most appropriate information based on their emotions.

[0283] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0284] Step 1:

[0285] Data collection

[0286] The server connects to the application programming interfaces (APIs) of various information sources used by the user and collects data. It receives authentication from each information source using an API key or OAuth token as input. For example, it retrieves the user's schedule data from the Google Calendar API and email data from the Gmail API. As output, it stores the collected data in JSON format on the server.

[0287] Specifically, the server sends the following API request:

[0288] GET https: / / www.googleapis.com / calendar / v3 / calendars / primary / events?key=API_KEY

[0289] This request returns the event data from Google Calendar.

[0290] Step 2:

[0291] Data organization and integration

[0292] The server organizes and consolidates the collected data. It uses data from each source stored in JSON format as input. It standardizes the data format, deletes unnecessary data, and eliminates duplicate data. Specifically, it uses Python's Pandas and NumPy libraries to handle the data as a data frame, extracts and formats the necessary information, and generates a consistent dataset as output.

[0293] Specifically, the server performs the following operations:

[0294] import pandas as pd

[0295] calendar_data = pd.read_json('calendar_data.json')

[0296] email_data = pd.read_json('email_data.json')

[0297] merged_data = pd.merge(calendar_data, email_data, on='event_id', how='outer')

[0298] This will merge your calendar and email data.

[0299] Step 3:

[0300] emotion recognition

[0301] The emotion engine recognizes the user's emotions. As input, it uses the user's text data, voice data, and facial expression data. It sends the data to an emotion recognition API to perform emotion analysis. For example, it uses Microsoft's Emotion API to obtain the emotional state from the text data. As output, an emotion score is returned to the server.

[0302] Specifically, the server sends the following API request:

[0303] POST https: / / westus.api.cognitive.microsoft.com / text / analytics / v3.0 / sentiment

[0304] This request returns a sentiment score for the text data.

[0305] Step 4:

[0306] Data Analysis and Personalization

[0307] The server uses machine learning algorithms to analyze the data and generate personalized information. It uses the cleaned and consolidated data and sentiment scores as input. It applies predictive models based on historical data and sentiment data using TensorFlow or PyTorch. As output, it generates personalized information tailored to the user's needs.

[0308] Specifically, the server performs the following operations:

[0309] import tensorflow as tf

[0310] model = tf.keras.models.load_model('model.h5')

[0311] predictions = model.predict(merged_data)

[0312] This operation generates personalized information.

[0313] Step 5:

[0314] Providing data to users

[0315] The device provides personalized information through a user interface. It uses the parsed personalized information as input, provides it to the user as notifications or messages through a smartphone application or web interface, and displays the information the user needs as output.

[0316] Specifically, the device performs the following operations:

[0317] NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);

[0318] Notification notification = new Notification.Builder(this)

[0319] .setContentTitle("Relaxation Guide")

[0320] .setContentText("Click here for a 10-minute guided meditation to help you relax right now")

[0321] .setSmallIcon(R.drawable.ic_notification)

[0322] .build();

[0323] notificationManager.notify(notificationId, notification);

[0324] This action will cause a notification to appear on the user's device.

[0325] Through the above steps, a system is realized that allows users to integrate data from a variety of information sources and provide optimal information according to their emotions.

[0326] (Application example 2)

[0327] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0328] In industrial environments, it is necessary to collect and integrate data from multiple different platforms and systems to obtain accurate information. However, there is a lack of mechanisms for efficiently collecting and integrating individual data, resulting in issues such as data duplication and the cumbersome task of organizing it. Furthermore, there is a lack of personalized support based on the emotions of workers, which prevents sufficient improvements in production efficiency and safety.

[0329] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and providing personalized information according to the user's needs and emotions; means for generating suggestions for improving efficiency and safety in the work environment based on the integrated data; means for providing the suggestions to the user through a user interface and providing feedback in real time; and means for recognizing the user's emotions and reflecting the emotional information in the analyzed data. This makes it possible to efficiently collect data from multiple data sources and support real-time improvements in the efficiency and safety of industrial environments based on the integrated information.

[0330] A "platform API" is an interface for exchanging and manipulating data between different systems and devices.

[0331] "Means of collecting data" refers to a mechanism for obtaining the necessary information through APIs of different platforms.

[0332] "Means for organizing and integrating data" refers to a mechanism for systematically organizing collected information, eliminating duplicate data, and consolidating it into a single data set.

[0333] A "machine learning model" is an algorithm for analyzing large amounts of data and finding patterns, and is a model for making adaptive predictions and suggestions based on the results obtained from the analysis.

[0334] "Personalized information" is information that is customized according to the individual needs and feelings of the user.

[0335] The "means for generating proposals" is a mechanism for creating specific actions and proposals for users based on the analyzed data.

[0336] A "user interface" is a screen or operating means that allows a user to exchange information with a system.

[0337] "Means of providing feedback in real time" refers to a mechanism that instantly notifies the user of analysis results and suggestions, and responds immediately to the user's actions and choices.

[0338] A "means for recognizing emotions" is a mechanism for analyzing emotions from the user's facial expressions, voice, etc., and converting those emotions into data.

[0339] The present invention relates to a data integration management system for improving efficiency and safety in an industrial environment. The system is implemented using the following hardware and software.

[0340] System Configuration

[0341] The system includes the following components:

[0342] 1. Data collection server: Connects to APIs of different platforms to collect data. Specific API examples include factory sensor device APIs and surveillance camera APIs.

[0343] 2. Data integration server: Organizes the collected data, eliminates duplicates, and integrates them. A database such as MongoDB is used to generate an integrated data set.

[0344] 3. Emotion recognition engine: Recognizes emotions from the user's facial expressions and voice, and generates emotional data. Specifically, it uses the Emotion API.

[0345] 4. Analysis Server: Analyzes the integrated data using machine learning models (e.g., built using TensorFlow or PyTorch) and generates personalized information according to the user's needs and emotions.

[0346] 5. User Interface: Provide feedback to the robot using the Robot Operating System (ROS).

[0347] Processing Description

[0348] The data collection server collects data via API from factory sensor devices, surveillance cameras, inventory systems, etc. This allows the real-time data obtained from each platform to be integrated.

[0349] The data integration server organizes the collected data, eliminating duplicates and generating a consistent dataset, which is stored in MongoDB for efficient data manipulation.

[0350] The emotion recognition engine analyzes the user's facial expressions and voice data acquired from a camera or microphone to generate emotion data. This emotion data is added to the integrated data and reflected in the analysis.

[0351] The analytics server runs machine learning models using the integrated data and sentiment data to generate recommendations to improve production efficiency and safety, including real-time work efficiency improvements, break recommendations, and inventory reordering.

[0352] The user interface uses ROS to provide specific feedback, such as prompting workers to take a break or showing instructions on how to avoid hazards.

[0353] Specific examples

[0354] 1. While a user is working in a factory, a camera captures the user's facial expression data.

[0355] 2. The data collection server collects sensor data and surveillance camera video data via API.

[0356] 3. The data integration server organizes this data and generates a consistent dataset.

[0357] 4. The emotion recognition engine recognizes the user's emotions and adds them to the integrated data.

[0358] 5. The analytics server analyzes the data using machine learning models to generate personalized recommendations.

[0359] 6. The user interface provides real-time feedback to the user through ROS.

[0360] Prompt Sentence Examples

[0361] Integrate factory sensor data, surveillance camera footage, inventory data, and emotion data obtained from workers' facial expressions, analyze the data using machine learning models, and generate suggestions to improve production efficiency and work safety in the factory. Specifically, provide feedback including actions such as suggesting a break if a worker is tired, or automating reordering if inventory is low.

[0362] This system can efficiently collect data from multiple sources and use the analysis results to help improve the efficiency and safety of industrial environments in real time, which is expected to improve factory productivity and worker satisfaction.

[0363] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0364] Step 1:

[0365] Data collection

[0366] The server collects data from APIs such as factory sensor devices, surveillance cameras, and inventory systems. Although the data format obtained from each device is different, it accesses each API endpoint to obtain sensor information, video data, and inventory information. The input to this step is the raw data obtained from the API, and the output is a temporarily stored dataset. Specifically, it sends an HTTP request to the API endpoint and obtains the response data.

[0367] Step 2:

[0368] Data Integration

[0369] The server integrates the collected data in different formats and eliminates duplicates. To do this, it organizes the data using a database such as MongoDB. The input to this step is the collected raw data, and the output is an integrated dataset. Specifically, it reads the data stored in the database, matches the data structure, and performs a process to remove duplicate items.

[0370] Step 3:

[0371] emotion recognition

[0372] The server inputs image data acquired from the surveillance camera into an emotion recognition engine to analyze the user's emotions. This engine uses the Emotion API to analyze emotions from the image data and output it as data. The input for this step is image data, and the output is analyzed emotion data. Specifically, the image data is sent to the Emotion API and emotion-tagged data is received.

[0373] Step 4:

[0374] Data analysis

[0375] The server analyzes the integrated data and sentiment data using machine learning models such as TensorFlow and PyTorch. This analysis generates personalized suggestions based on the user's needs and sentiment. The inputs to this step are the integrated dataset and sentiment data, and the output is personalized suggestions. Specifically, the data is preprocessed, input to the machine learning model, and the analysis results are obtained.

[0376] Step 5:

[0377] Proposal Generation

[0378] The server generates proposals to improve the factory's production efficiency and safety based on the results of the data analysis. The input to this step is the analysis results, and the output is specific proposals. Specific operations include formatting the proposals based on the analysis results and generating actions to improve work efficiency, propose breaks, and reorder inventory.

[0379] Step 6:

[0380] Providing feedback

[0381] The server provides real-time feedback to the user through a user interface. Specifically, it uses the Robot Operating System (ROS) to notify the worker and prompt them to take specific actions. The input of this step is the generated suggestion, and the output is the feedback displayed to the user. The specific action is to communicate the suggestion to the user through ROS.

[0382] Step 7:

[0383] Monitoring User Actions

[0384] The device monitors how the user responds to the feedback. This accumulates data that can be used for future analysis. The input for this step is the user's behavioral data, and the output is saved as reaction data. Specifically, the device collects the user's behavioral log and saves it in a database that will be used for the next data analysis.

[0385] As a result, it is possible to efficiently collect information from multiple data sources and, based on the analysis results, to help improve the efficiency and safety of industrial environments in real time.

[0386] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0387] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0388] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0389] [Second embodiment]

[0390] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0391] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0392] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0393] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0394] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0395] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0396] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0397] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0398] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0399] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0400] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0401] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0402] This invention relates to a data integration management system that collects, organizes, and consolidates data through APIs from different platforms, and further analyzes the data using machine learning models to provide personalized information according to user needs.

[0403] System Configuration

[0404] The system mainly consists of the following components:

[0405] 1. Data collection server: Acquires data from different platforms via API.

[0406] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[0407] 3. Analysis Server: Based on the integrated data, it uses machine learning models to analyze the data and generate personalized information based on the user's needs.

[0408] 4. User Interface: The interface through which the user accesses personalized information.

[0409] Program processing

[0410] Data collection

[0411] The server connects to the APIs of the various platforms used by the user and collects data. For example, if a user uses cloud storage, a calendar app, or an email service, the server collects data from each API.

[0412] Data organization and integration

[0413] The server then organizes and consolidates the data collected from each platform, automatically eliminating duplicate data and unnecessary information. For example, calendar events and email content are consolidated and compiled into a single data set for each user.

[0414] Data Analysis and Personalization

[0415] The server then uses machine learning models to analyze the integrated data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set. For example, a businessperson might prioritize schedule changes and important emails.

[0416] Providing data to users

[0417] The device provides the analyzed personalized data to the user, and helps the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing the user to check important appointments and messages.

[0418] Specific examples

[0419] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0420] 2. The device sends requests to the server and collects data from different platforms.

[0421] 3. The server collects data from cloud storage, calendar, and email services through APIs.

[0422] 4. The server organizes and consolidates the collected data to create a coherent data set.

[0423] 5. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs.

[0424] 6. The device provides personalized information to the user through a user interface.

[0425] 7. Users can view schedules and important emails generated on their smartphone screens and take action to improve their productivity.

[0426] As described above, the system of the present invention efficiently integrates data from various devices and cloud services, providing an environment in which users can quickly access the information they need. This system is expected to significantly reduce the data management burden on users, especially business people.

[0427] The processing flow will be explained below.

[0428] Step 1:

[0429] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[0430] Step 2:

[0431] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[0432] Step 3:

[0433] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services) to collect data. The server sends an HTTP request to the service's API endpoint and stores the collected data in memory.

[0434] Step 4:

[0435] The server organizes and integrates the collected data, automatically eliminating duplicate data and unnecessary information. For example, it aligns file information retrieved from cloud storage with calendar appointment information to create a single integrated data set.

[0436] Step 5:

[0437] The server then uses machine learning models to analyze the combined data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set.

[0438] Step 6:

[0439] The server sends the parsed personalized data to the device, formats the data in the specified format (e.g., JSON), and sends it to the device.

[0440] Step 7:

[0441] The device displays the received data through a user interface. The device displays personalized information (e.g., the latest schedule, important email notifications) on the application screen.

[0442] Step 8:

[0443] The user can review the provided data and take necessary actions. For example, the user can review the displayed schedule details, prepare for a meeting, or instantly reply to an important email.

[0444] Example 1

[0445] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0446] Conventional data collection and integration systems did not efficiently collect and organize data from different platforms, and duplicate and unnecessary data was not adequately eliminated. Furthermore, they lacked personalized functionality to enable users to quickly obtain the information they needed, reducing the efficiency of data management and information retrieval. This increased the burden of information management on users and led to problems with insufficient data utilization.

[0447] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0448] In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and personalizing it according to the user's needs; means for the terminal to send a data integration request to the server based on the user's operation; means for eliminating unnecessary and redundant information from the integrated data; and means for sending the generated personalized data to the terminal and displaying it on a user interface. This allows data from different platforms to be efficiently collected and organized, and redundant and unnecessary data to be eliminated, enabling users to quickly obtain the information they need. Furthermore, the personalization function reduces the user's burden of information management and improves data utilization.

[0449] "Different platforms" refers to different types of services and applications, such as cloud storage, calendar applications, and email services.

[0450] "API" stands for Application Programming Interface, an interface that allows software to communicate with other software.

[0451] "Means of collecting data" refers to the ability to obtain data through APIs of different platforms.

[0452] "Means for organizing and integrating data" refers to the function of organizing collected data, eliminating duplicate and unnecessary data, and generating a unified data set.

[0453] A "machine learning model" is an algorithm or model that analyzes data and personalizes information based on user behavior and needs.

[0454] "Means for personalization" refers to the function of customizing and providing information according to the needs and preferences of individual users.

[0455] The "means for the terminal to send a data integration request to the server based on a user's operation" is a function in which the user performs an operation through the device and sends a data integration request to the server based on that operation.

[0456] "Means for eliminating unnecessary and redundant information from integrated data" refers to a function that organizes and integrates collected data and eliminates redundant and unnecessary information.

[0457] "Means for transmitting the generated personalized data to the terminal and displaying it on the user interface" refers to a function for transmitting personalized information generated based on the analyzed data to the terminal and displaying it on the interface used by the user.

[0458] A "generative AI model" is an algorithm or model that uses artificial intelligence to perform generative tasks, particularly in data analysis and personalization.

[0459] A "prompt" is a natural language sentence input to a generative AI model to specify what the model should do.

[0460] The present invention relates to a data integration management system that collects data through APIs of different platforms and organizes and consolidates them. The system further analyzes the data using machine learning models and provides personalized information according to the user's needs. Specific embodiments of the present invention are described below.

[0461] System Configuration

[0462] The system mainly consists of the following components:

[0463] 1. Data collection server: Acquires data from different platforms via APIs, such as Google Calendar API, Gmail API, and cloud storage API.

[0464] 2. Data integration server: Organizes and integrates collected data, eliminating duplicate and unnecessary data.

[0465] 3. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models (e.g., TensorFlow, Scikit-learn) to generate personalized information based on the user's needs.

[0466] 4. User Interface: The interface through which users access personalized information. Examples include smartphone applications and web interfaces.

[0467] Program processing

[0468] Data collection

[0469] The server connects to the APIs of the various platforms used by the user and collects data. For example, if the user uses cloud storage, a calendar app, or an email service, the server collects data from each API (e.g., Google Calendar API, Gmail API, Dropbox API).

[0470] Data organization and integration

[0471] The server organizes and consolidates the data collected from each platform. During this process, duplicate data and unnecessary information are automatically eliminated. For example, if the same email exists on multiple devices, it will be consolidated into one. Duplicate calendar events will also be consolidated.

[0472] Data Analysis and Personalization

[0473] The server analyzes the integrated data using machine learning models (e.g., TensorFlow and Scikit-learn). The goal of the analysis is to understand the user's behavioral patterns and needs and personalize the data accordingly. For example, it can prioritize the most important meetings and tasks in a business person's schedule.

[0474] Providing data to users

[0475] The server then sends the generated personalized data to the device and provides it to the user through a user interface. For example, a notification may be displayed in a smartphone app, allowing the user to check it. Based on the user's request, the server also uses the generative AI model to extract and provide information related to the prompt sentence.

[0476] Specific examples

[0477] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0478] 2. The device sends requests to the server and collects data from different platforms.

[0479] 3. The server connects to the Google Calendar API, Gmail API, and Dropbox API to retrieve data from each service.

[0480] 4. The server analyzes the data and deletes and organizes duplicates. If there are emails with the same subject but different text, they are merged into one.

[0481] 5. The server uses machine learning models to analyze the data, extract key information relevant to the user's needs, and generate a personalized dataset.

[0482] 6. The device receives the data sent from the server and provides it to the user through a user interface. For example, important schedule or email notifications are displayed as app icons or on the screen.

[0483] 7. The user checks their schedule and important emails on their smartphone screen and takes appropriate action.

[0484] Prompt Sentence Examples

[0485] For example, by entering "What is my schedule for this week?", personalized schedule information will be displayed. By utilizing this prompt, users can obtain the information they need in natural language.

[0486] As a result, the present invention provides a system that efficiently collects and organizes data from different platforms and provides important information personalized based on user needs, thereby reducing the burden of information management on users and improving data utilization.

[0487] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0488] Step 1:

[0489] The user launches the app on their smartphone and taps the "Integrate Data" button.

[0490] Input: User action (tapping the "Integrate data" button)

[0491] Output: A data integration request is sent to the server

[0492] Step 2:

[0493] The terminal transmits a data collection request to the server based on the user's operation.

[0494] Input: Requests made by the user

[0495] Output: A data collection request is sent to the server

[0496] Step 3:

[0497] Based on the received request, the server accesses the APIs of various platforms to collect data, such as the Google Calendar API, Gmail API, and cloud storage API.

[0498] Input: Data Collection Request

[0499] Output: Data collected from each platform (e.g., calendar events, emails, files in storage)

[0500] Step 4:

[0501] The server then organizes and consolidates the collected data, a process that converts data from different formats into a common format and eliminates duplicate data and unnecessary information.

[0502] Input: Data collected from each platform

[0503] Output: Cleaned and consolidated dataset

[0504] Step 5:

[0505] The server inputs the integrated data into machine learning models (e.g., TensorFlow, Scikit-learn) and analyzes the data based on user behavior patterns and needs.

[0506] Input: Cleaned and consolidated dataset

[0507] Output: Analysis results (personalized information according to user needs)

[0508] Step 6:

[0509] The server uses the analysis to generate a personalized data set, including important emails for the user and reminders for upcoming meetings.

[0510] Input: Analysis results

[0511] Output: A personalized dataset

[0512] Step 7:

[0513] The server transmits the generated personalized data to the terminal and instructs the terminal to display the data on the user interface.

[0514] Input: personalized dataset

[0515] Output: Personalized information displayed in the user interface (e.g. smartphone notifications)

[0516] Step 8:

[0517] The user can then take the next action based on the personalized information displayed on the device, such as replying to an important email or preparing to attend a scheduled meeting.

[0518] Input: Personalization information displayed in the user interface

[0519] Output: User's next action (e.g., replying to an email, preparing for a meeting)

[0520] This is the specific processing flow of this system. This system enables users to manage information efficiently, greatly improving the utilization of data.

[0521] (Application example 1)

[0522] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0523] In modern society, it is extremely important to collect data from multiple platforms and devices and manage it centrally. However, integrating and organizing data from different platforms is not easy. Furthermore, advanced data analysis and personalization technologies are required to quickly provide users with the information they need from a wide range of information. Furthermore, it is also important to present personalized information in a visually easy-to-understand manner. The present invention has been proposed to solve these problems.

[0524] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0525] In this invention, the server includes means for collecting data through APIs of different platforms, means for organizing and integrating the collected data, and means for analyzing the data using a machine learning model and personalizing it according to the user's needs. This allows the data collected from different platforms to be organized and integrated, and the information required by the user to be personalized through analysis by the machine learning model, and to be visually provided via a display device.

[0526] "Distinct platforms" refers to multiple software and service offerings, each operated by a different vendor and using a different technology stack.

[0527] "API" stands for Application Program Interface, a set of rules and tools for exchanging data and functions between software programs.

[0528] "Means of collecting data" refers to the technology and devices used to obtain information from different platforms via APIs and aggregate it centrally.

[0529] "Data organization and integration methods" are techniques and processes that organize collected data in a consistent and coherent manner, eliminating duplicate and unnecessary information.

[0530] A "machine learning model" is a computer program that uses algorithms and statistical methods to analyze data and make predictions or recognize patterns.

[0531] "Personalization tools" are technologies and methods that customize collected and integrated data to meet the specific needs of users.

[0532] A "display device" is a device for presenting visual information to a user, such as smart glasses, a tablet, or a smartphone.

[0533] A "visual display" is a screen or projector for visually displaying digital information.

[0534] The "means of providing to the user" refers to the technology and protocols that appropriately transmit and present analyzed and personalized information to the device used by the user.

[0535] An embodiment of the present invention will be described.

[0536] The system of the present invention collects data through APIs of different platforms, organizes and integrates the collected data, analyzes it using machine learning models, and provides personalized information based on user needs using a visual display. The main components include a data collection server, a data integration server, an analysis server, and a display device such as smart glasses.

[0537] First, the server connects to the APIs of various platforms to collect user activity data, purchase history, etc. The collected data is sent to a data integration server, where it is organized and de-duplicated. The organized data is then sent to an analysis server, where it is analyzed using machine learning models. The analysis results are presented to the user as personalized information optimized for them. This information is presented via a visual display such as smart glasses.

[0538] Hardware and software used

[0539] Hardware:

[0540] Smart glasses (e.g. Google Glass, Vuzix Blade)

[0541] Data Collection Server

[0542] Data Integration Server

[0543] Analysis Server

[0544] Customer Relationship Management System (CRM)

[0545] software:

[0546] API connection libraries (e.g., Postman, Retrofit)

[0547] Machine learning libraries (e.g. TensorFlow, scikit-learn)

[0548] Database (e.g. MySQL, MongoDB)

[0549] UI development frameworks (e.g., Angular, React)

[0550] Processing flow and example

[0551] First, when a user visits the store, the server collects data via APIs from different platforms, such as cloud storage, calendar services, and email services. For example, information about the user's past purchase history and browsing history is obtained. The collected data is then organized and integrated by a data integration server. During this process, duplicate and unnecessary data is eliminated, creating a consistent data set.

[0552] The analytics server then uses machine learning models based on this integrated data to analyze the information needed to meet the user's needs and generate personalized information, such as recommendations for new camping equipment based on the user's previous purchases, or new product recommendations.

[0553] Finally, the generated personalized information is provided to the user through a visual display such as smart glasses, allowing store clerks to make more accurate product recommendations to customers, which is expected to improve customer satisfaction.

[0554] Specific prompt examples

[0555] Provide information to the machine learning model using prompt statements such as:

[0556] Prompt statement:

[0557] "Please recommend some products to a male customer in his 30s who enjoys the outdoors and has purchased a lot of camping equipment in the past year."

[0558] Example response:

[0559] "I would like to recommend a new camping stove and waterproof tent to this customer. I can also offer a great deal on our recently arrived multi-purpose knife."

[0560] In this way, the system of the present invention can efficiently collect and integrate data from different platforms, use machine learning models to personalize information based on the user's needs, and provide it through a visual display, thereby improving the user experience.

[0561] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0562] Step 1:

[0563] The server connects to the APIs of different platforms (e.g., cloud storage, calendar services, email services) to collect user activity data, purchase history, etc. Specifically, it sends an HTTP request to each API endpoint and retrieves the returned data. The input to this step is the API endpoint of each platform and user authentication information, and the output is raw data of the user's activity data and purchase history.

[0564] Step 2:

[0565] The server sends the collected data to the data integration server, which organizes and integrates the data. Specifically, it standardizes the data format and eliminates duplicate data and unnecessary information. This process involves merging, for example, duplicate purchase histories or identical event information into one. The input to this step is the raw data collected from each platform, and the output is an organized and integrated dataset.

[0566] Step 3:

[0567] The server sends the consolidated data to the analysis server, where it is analyzed using a machine learning model. Specifically, data analysis is performed to predict the user's interests and preferences based on past purchase data and activity history. This process recommends products and services that best suit the user's needs. The input to this step is the consolidated data set, and the output is personalized recommendations for the user.

[0568] Step 4:

[0569] The server then transmits the analyzed personalized information to a display device (e.g., smart glasses) to provide it to the user. Specifically, the server converts the generated recommendation information into a format suitable for display on the device screen and transmits it to the smart glasses. The input of this step is the personalized information generated by the machine learning model, and the output is the recommendation information displayed on the visual display.

[0570] Step 5:

[0571] The user checks the personalized information presented through the smart glasses and takes appropriate action. Specifically, the system explains recommended products and services to the customer and helps promote purchases. The input of this step is the recommended information displayed on the smart glasses, and the output is the user's action (e.g., product suggestions or purchasing procedures).

[0572] Through each of the above steps, data collected from different platforms is efficiently integrated, and personalized information is provided to users based on the results of analysis using machine learning models, thereby achieving advanced customer service and improved service.

[0573] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0574] This invention relates to a data integration management system that collects, organizes, and integrates data through APIs from different platforms. Furthermore, it analyzes the data using machine learning models and provides personalized information according to the user's needs and emotions. By combining it with an emotion engine that recognizes the user's emotions, even more advanced personalization becomes possible.

[0575] System Configuration

[0576] The system mainly consists of the following components:

[0577] 1. Data collection server: Acquires data from different platforms via API.

[0578] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[0579] 3. Emotion engine: Recognizes user emotions and reflects them in data analysis.

[0580] 4. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models to generate personalized information based on the user's needs and emotions.

[0581] 5. User Interface: The interface through which the user accesses personalized information.

[0582] Program processing

[0583] Data collection

[0584] The server connects to the APIs of the various platforms the user uses and collects data. For example, if the user uses cloud storage, a calendar app, an email service, or a social networking service, the server collects data from each API.

[0585] Data organization and integration

[0586] The server organizes and integrates the data collected from each platform. In doing so, it automatically eliminates duplicate data and unnecessary information. For example, it aligns posts obtained from social media with communications from email services and provides them as an integrated data set.

[0587] emotion recognition

[0588] The emotion engine recognizes the user's emotions. It analyzes user input, voice data, facial expression data, etc. to identify the emotion the user is currently feeling. This emotion data is reflected in subsequent data analysis and personalization.

[0589] Data Analysis and Personalization

[0590] The server then analyzes the integrated data using machine learning models to extract the most relevant information based on the user's past behavior, current situation, and perceived emotions, generating a personalized data set. For example, if the user is feeling stressed, it will prioritize relaxation information and entertainment content.

[0591] Providing data to users

[0592] The device provides the analyzed personalized data to the user, helping the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing users to check important schedules, messages, and emotional content.

[0593] Specific examples

[0594] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0595] 2. The device sends a request to the server to collect data from different platforms.

[0596] 3. The server collects data from cloud storage, calendars, email services, and social media.

[0597] 4. The server consolidates and organizes the collected data to create a consistent data set.

[0598] 5. The emotion engine recognizes the user's emotions and provides the data to the server.

[0599] 6. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs and perceived emotions.

[0600] 7. The device provides personalized information to the user through a user interface.

[0601] 8. Users can view schedules, important emails, and emotional suggestions (such as entertainment content or relaxation methods) generated on their smartphone screen, and take action to improve productivity.

[0602] As described above, the system of the present invention efficiently integrates data from a variety of devices and cloud services, providing an environment in which users can quickly access the information they need. In particular, by combining it with an emotion engine, advanced personalization based on the user's emotional state can be achieved, making it possible to better meet individual needs. This system is expected to significantly reduce the data management burden on users, particularly business people, and improve the quality of their daily lives.

[0603] The processing flow will be explained below.

[0604] Step 1:

[0605] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[0606] Step 2:

[0607] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[0608] Step 3:

[0609] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services, SNS) to collect data. The server sends an HTTP request to the service's API endpoint and temporarily stores the collected data in memory.

[0610] Step 4:

[0611] The server organizes and integrates the data it collects. For example, it combines file data collected from cloud storage and schedule data collected from a calendar app into a single integrated data set. It also eliminates duplicate and unnecessary data.

[0612] Step 5:

[0613] The emotion engine recognizes the user's emotions. The user's voice data, text data, or image data obtained from the terminal or external device is sent to the server, which inputs it into the emotion engine. The emotion engine analyzes this data and recognizes the user's emotional state (e.g., joy, sadness, stress).

[0614] Step 6:

[0615] The server uses machine learning models to analyze the integrated data. The server analyzes the data based on the user's past behavioral history, current situation, and recognized emotions, and extracts the information most appropriate for the user. For example, if the user is feeling stressed, it will prioritize information related to relaxation.

[0616] Step 7:

[0617] The server sends the parsed personalized data to the device. The server formats the data in the specified format (e.g., JSON) and sends it to the device. The data includes important schedules, emails, SNS notifications, etc.

[0618] Step 8:

[0619] The device displays the received data through a user interface, and the device displays personalized information (e.g., the latest schedule, important email notifications, and emotional entertainment content) on the application screen.

[0620] Step 9:

[0621] The user can review the provided data and take necessary actions. For example, the user can check the displayed schedule details and prepare for a meeting. The user can also reduce stress by viewing relaxing content tailored to the user's emotions.

[0622] As described above, this system collects and integrates data from the user's various devices and cloud services, and by using an emotion engine, achieves advanced personalization according to the user's emotional state, allowing users to quickly access the information they need most and supporting efficient daily life and work performance.

[0623] Example 2

[0624] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0625] Currently, collecting, integrating, and analyzing data from different sources is a time-consuming and labor-intensive task. It is also difficult to provide personalized information that takes into account the user's emotions, making it difficult to provide appropriate services that meet the user's needs. Furthermore, issues such as data duplication and organization are not taken into consideration, resulting in a lack of user convenience.

[0626] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data through application programming interfaces of different information sources, means for organizing and integrating the collected data, means for eliminating duplicate data, means for using an emotion engine to recognize the user's emotions, means for analyzing the data using a machine learning algorithm and personalizing it according to the user's needs and emotions, and means for providing the personalized data to the user. This makes it possible to efficiently collect and integrate data from multiple information sources and provide personalized information that takes the user's emotions into consideration.

[0627] "Sources" refer to the systems and platforms that provide data, including cloud storage, calendar apps, email services, and social networking services.

[0628] An "application programming interface" is a defined set of communication rules that allow software applications to interact with other applications and services.

[0629] "Data collection" refers to the process of obtaining data from different sources and storing it on a server.

[0630] Data consolidation is the process of organizing collected data into a consistent format, linking it where necessary, and eliminating duplicate data.

[0631] "Duplicate data elimination" is the process of detecting and removing duplicate data to prevent the same information from being collected multiple times.

[0632] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions, text, voice, etc. to recognize emotions.

[0633] A "machine learning algorithm" is a mathematical model or method for learning patterns from large amounts of data and making predictions or classifications.

[0634] "Personalization" is the process of providing information optimized to the needs and emotions of individual users based on collected data and analysis results.

[0635] "Means for providing to the user" refers to an interface or notification system for providing the analyzed information in a form that the user can use.

[0636] This invention is a system that collects data through application programming interfaces (APIs) from different information sources, organizes, integrates, and analyzes them, and provides personalized information according to the user's needs and emotions. The main components of the system include the following:

[0637] 1. Data collection server: The server connects to the APIs of cloud storage, calendar apps, email services, social networking sites, etc. used by users to collect data. Specifically, it uses APIs such as Google Drive, Google Calendar, Gmail, and Twitter.

[0638] 2. Data integration server: The server integrates the collected data and organizes it into a consistent format. This process also eliminates duplicate data. Specifically, the server processes and formats the data using the Python libraries Pandas and NumPy.

[0639] 3. Emotion Engine: The emotion engine is designed to recognize the user's emotions. The server uses an emotion recognition API (e.g., Emotion API) to analyze the text data, voice data, and facial expression data entered by the user.

[0640] 4. Analytics Server: The server analyzes the data using machine learning algorithms to generate personalized information based on the user's past behavior, current situation, and perceived emotions. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch.

[0641] 5. User Interface: The device is responsible for providing personalized information to the user. It displays the information as notifications and messages through smartphone applications and web interfaces, making it easily accessible to the user.

[0642] For example, when a user launches a dedicated smartphone app and taps the "Integrate Data" button, the device sends a data collection request to the server. The server accesses various APIs to collect, organize, and integrate the necessary data. The emotion engine then analyzes the text and facial expression data to identify the user's emotions. The data is analyzed using machine learning algorithms to generate personalized information, which the device then provides to the user as notifications or messages. This series of processes allows users to quickly access the information they need and reduces the burden of information management.

[0643] An example prompt is:

[0644] "I want to develop a system that integrates and organizes data from different platforms, recognizes user emotions, and provides analytical data. I will use EmotionAPI for emotion recognition and TensorFlow for machine learning. Please explain the program process for this system step by step."

[0645] This system is expected to improve the quality of daily life and business by enabling users to integrate data from a variety of sources and obtain the most appropriate information based on their emotions.

[0646] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0647] Step 1:

[0648] Data collection

[0649] The server connects to the application programming interfaces (APIs) of various information sources used by the user and collects data. It receives authentication from each information source using an API key or OAuth token as input. For example, it retrieves the user's schedule data from the Google Calendar API and email data from the Gmail API. As output, it stores the collected data in JSON format on the server.

[0650] Specifically, the server sends the following API request:

[0651] GET https: / / www.googleapis.com / calendar / v3 / calendars / primary / events?key=API_KEY

[0652] This request returns the event data from Google Calendar.

[0653] Step 2:

[0654] Data organization and integration

[0655] The server organizes and consolidates the collected data. It uses data from each source stored in JSON format as input. It standardizes the data format, deletes unnecessary data, and eliminates duplicate data. Specifically, it uses Python's Pandas and NumPy libraries to handle the data as a data frame, extracts and formats the necessary information, and generates a consistent dataset as output.

[0656] Specifically, the server performs the following operations:

[0657] import pandas as pd

[0658] calendar_data = pd.read_json('calendar_data.json')

[0659] email_data = pd.read_json('email_data.json')

[0660] merged_data = pd.merge(calendar_data, email_data, on='event_id', how='outer')

[0661] This will merge your calendar and email data.

[0662] Step 3:

[0663] emotion recognition

[0664] The emotion engine recognizes the user's emotions. As input, it uses the user's text data, voice data, and facial expression data. It sends the data to an emotion recognition API to perform emotion analysis. For example, it uses Microsoft's Emotion API to obtain the emotional state from the text data. As output, an emotion score is returned to the server.

[0665] Specifically, the server sends the following API request:

[0666] POST https: / / westus.api.cognitive.microsoft.com / text / analytics / v3.0 / sentiment

[0667] This request returns a sentiment score for the text data.

[0668] Step 4:

[0669] Data Analysis and Personalization

[0670] The server uses machine learning algorithms to analyze the data and generate personalized information. It uses the cleaned and consolidated data and sentiment scores as input. It applies predictive models based on historical data and sentiment data using TensorFlow or PyTorch. As output, it generates personalized information tailored to the user's needs.

[0671] Specifically, the server performs the following operations:

[0672] import tensorflow as tf

[0673] model = tf.keras.models.load_model('model.h5')

[0674] predictions = model.predict(merged_data)

[0675] This operation generates personalized information.

[0676] Step 5:

[0677] Providing data to users

[0678] The device provides personalized information through a user interface. It uses the parsed personalized information as input, provides it to the user as notifications or messages through a smartphone application or web interface, and displays the information the user needs as output.

[0679] Specifically, the device performs the following operations:

[0680] NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);

[0681] Notification notification = new Notification.Builder(this)

[0682] .setContentTitle("Relaxation Guide")

[0683] .setContentText("Click here for a 10-minute guided meditation to help you relax right now")

[0684] .setSmallIcon(R.drawable.ic_notification)

[0685] .build();

[0686] notificationManager.notify(notificationId, notification);

[0687] This action will cause a notification to appear on the user's device.

[0688] Through the above steps, a system is realized that allows users to integrate data from a variety of information sources and provide optimal information according to their emotions.

[0689] (Application example 2)

[0690] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0691] In industrial environments, it is necessary to collect and integrate data from multiple different platforms and systems to obtain accurate information. However, there is a lack of mechanisms for efficiently collecting and integrating individual data, resulting in issues such as data duplication and the cumbersome task of organizing it. Furthermore, there is a lack of personalized support based on the emotions of workers, which prevents sufficient improvements in production efficiency and safety.

[0692] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and providing personalized information according to the user's needs and emotions; means for generating suggestions for improving efficiency and safety in the work environment based on the integrated data; means for providing the suggestions to the user through a user interface and providing feedback in real time; and means for recognizing the user's emotions and reflecting the emotional information in the analyzed data. This makes it possible to efficiently collect data from multiple data sources and support real-time improvements in the efficiency and safety of industrial environments based on the integrated information.

[0693] A "platform API" is an interface for exchanging and manipulating data between different systems and devices.

[0694] "Means of collecting data" refers to a mechanism for obtaining the necessary information through APIs of different platforms.

[0695] "Means for organizing and integrating data" refers to a mechanism for systematically organizing collected information, eliminating duplicate data, and consolidating it into a single data set.

[0696] A "machine learning model" is an algorithm for analyzing large amounts of data and finding patterns, and is a model for making adaptive predictions and suggestions based on the results obtained from the analysis.

[0697] "Personalized information" is information that is customized according to the individual needs and feelings of the user.

[0698] The "means for generating proposals" is a mechanism for creating specific actions and proposals for users based on the analyzed data.

[0699] A "user interface" is a screen or operating means that allows a user to exchange information with a system.

[0700] "Means of providing feedback in real time" refers to a mechanism that instantly notifies the user of analysis results and suggestions, and responds immediately to the user's actions and choices.

[0701] A "means for recognizing emotions" is a mechanism for analyzing emotions from the user's facial expressions, voice, etc., and converting those emotions into data.

[0702] The present invention relates to a data integration management system for improving efficiency and safety in an industrial environment. The system is implemented using the following hardware and software.

[0703] System Configuration

[0704] The system includes the following components:

[0705] 1. Data collection server: Connects to APIs of different platforms to collect data. Specific API examples include factory sensor device APIs and surveillance camera APIs.

[0706] 2. Data integration server: Organizes the collected data, eliminates duplicates, and integrates them. A database such as MongoDB is used to generate an integrated data set.

[0707] 3. Emotion recognition engine: Recognizes emotions from the user's facial expressions and voice, and generates emotional data. Specifically, it uses the Emotion API.

[0708] 4. Analysis Server: Analyzes the integrated data using machine learning models (e.g., built using TensorFlow or PyTorch) and generates personalized information according to the user's needs and emotions.

[0709] 5. User Interface: Provide feedback to the robot using the Robot Operating System (ROS).

[0710] Processing Description

[0711] The data collection server collects data via API from factory sensor devices, surveillance cameras, inventory systems, etc. This allows the real-time data obtained from each platform to be integrated.

[0712] The data integration server organizes the collected data, eliminating duplicates and generating a consistent dataset, which is stored in MongoDB for efficient data manipulation.

[0713] The emotion recognition engine analyzes the user's facial expressions and voice data acquired from a camera or microphone to generate emotion data. This emotion data is added to the integrated data and reflected in the analysis.

[0714] The analytics server runs machine learning models using the integrated data and sentiment data to generate recommendations to improve production efficiency and safety, including real-time work efficiency improvements, break recommendations, and inventory reordering.

[0715] The user interface uses ROS to provide specific feedback, such as prompting workers to take a break or showing instructions on how to avoid hazards.

[0716] Specific examples

[0717] 1. While a user is working in a factory, a camera captures the user's facial expression data.

[0718] 2. The data collection server collects sensor data and surveillance camera video data via API.

[0719] 3. The data integration server organizes this data and generates a consistent dataset.

[0720] 4. The emotion recognition engine recognizes the user's emotions and adds them to the integrated data.

[0721] 5. The analytics server analyzes the data using machine learning models to generate personalized recommendations.

[0722] 6. The user interface provides real-time feedback to the user through ROS.

[0723] Prompt Sentence Examples

[0724] Integrate factory sensor data, surveillance camera footage, inventory data, and emotion data obtained from workers' facial expressions, analyze the data using machine learning models, and generate suggestions to improve production efficiency and work safety in the factory. Specifically, provide feedback including actions such as suggesting a break if a worker is tired, or automating reordering if inventory is low.

[0725] This system can efficiently collect data from multiple sources and use the analysis results to help improve the efficiency and safety of industrial environments in real time, which is expected to improve factory productivity and worker satisfaction.

[0726] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0727] Step 1:

[0728] Data collection

[0729] The server collects data from APIs such as factory sensor devices, surveillance cameras, and inventory systems. Although the data format obtained from each device is different, it accesses each API endpoint to obtain sensor information, video data, and inventory information. The input to this step is the raw data obtained from the API, and the output is a temporarily stored dataset. Specifically, it sends an HTTP request to the API endpoint and obtains the response data.

[0730] Step 2:

[0731] Data Integration

[0732] The server integrates the collected data in different formats and eliminates duplicates. To do this, it organizes the data using a database such as MongoDB. The input to this step is the collected raw data, and the output is an integrated dataset. Specifically, it reads the data stored in the database, matches the data structure, and performs a process to remove duplicate items.

[0733] Step 3:

[0734] emotion recognition

[0735] The server inputs image data acquired from the surveillance camera into an emotion recognition engine to analyze the user's emotions. This engine uses the Emotion API to analyze emotions from the image data and output it as data. The input for this step is image data, and the output is analyzed emotion data. Specifically, the image data is sent to the Emotion API and emotion-tagged data is received.

[0736] Step 4:

[0737] Data analysis

[0738] The server analyzes the integrated data and sentiment data using machine learning models such as TensorFlow and PyTorch. This analysis generates personalized suggestions based on the user's needs and sentiment. The inputs to this step are the integrated dataset and sentiment data, and the output is personalized suggestions. Specifically, the data is preprocessed, input to the machine learning model, and the analysis results are obtained.

[0739] Step 5:

[0740] Proposal Generation

[0741] The server generates proposals to improve the factory's production efficiency and safety based on the results of the data analysis. The input to this step is the analysis results, and the output is specific proposals. Specific operations include formatting the proposals based on the analysis results and generating actions to improve work efficiency, propose breaks, and reorder inventory.

[0742] Step 6:

[0743] Providing feedback

[0744] The server provides real-time feedback to the user through a user interface. Specifically, it uses the Robot Operating System (ROS) to notify the worker and prompt them to take specific actions. The input of this step is the generated suggestion, and the output is the feedback displayed to the user. The specific action is to communicate the suggestion to the user through ROS.

[0745] Step 7:

[0746] Monitoring User Actions

[0747] The device monitors how the user responds to the feedback. This accumulates data that can be used for future analysis. The input for this step is the user's behavioral data, and the output is saved as reaction data. Specifically, the device collects the user's behavioral log and saves it in a database that will be used for the next data analysis.

[0748] As a result, it is possible to efficiently collect information from multiple data sources and, based on the analysis results, to help improve the efficiency and safety of industrial environments in real time.

[0749] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0750] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0751] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0752] [Third embodiment]

[0753] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0754] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0755] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0756] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0757] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0758] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0759] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0760] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0761] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0762] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0763] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0764] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0765] This invention relates to a data integration management system that collects, organizes, and consolidates data through APIs from different platforms, and further analyzes the data using machine learning models to provide personalized information according to user needs.

[0766] System Configuration

[0767] The system mainly consists of the following components:

[0768] 1. Data collection server: Acquires data from different platforms via API.

[0769] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[0770] 3. Analysis Server: Based on the integrated data, it uses machine learning models to analyze the data and generate personalized information based on the user's needs.

[0771] 4. User Interface: The interface through which the user accesses personalized information.

[0772] Program processing

[0773] Data collection

[0774] The server connects to the APIs of the various platforms used by the user and collects data. For example, if a user uses cloud storage, a calendar app, or an email service, the server collects data from each API.

[0775] Data organization and integration

[0776] The server then organizes and consolidates the data collected from each platform, automatically eliminating duplicate data and unnecessary information. For example, calendar events and email content are consolidated and compiled into a single data set for each user.

[0777] Data Analysis and Personalization

[0778] The server then uses machine learning models to analyze the integrated data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set. For example, a businessperson might prioritize schedule changes and important emails.

[0779] Providing data to users

[0780] The device provides the analyzed personalized data to the user, and helps the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing the user to check important appointments and messages.

[0781] Specific examples

[0782] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0783] 2. The device sends requests to the server and collects data from different platforms.

[0784] 3. The server collects data from cloud storage, calendar, and email services through APIs.

[0785] 4. The server organizes and consolidates the collected data to create a coherent data set.

[0786] 5. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs.

[0787] 6. The device provides personalized information to the user through a user interface.

[0788] 7. Users can view schedules and important emails generated on their smartphone screens and take action to improve their productivity.

[0789] As described above, the system of the present invention efficiently integrates data from various devices and cloud services, providing an environment in which users can quickly access the information they need. This system is expected to significantly reduce the data management burden on users, especially business people.

[0790] The processing flow will be explained below.

[0791] Step 1:

[0792] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[0793] Step 2:

[0794] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[0795] Step 3:

[0796] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services) to collect data. The server sends an HTTP request to the service's API endpoint and stores the collected data in memory.

[0797] Step 4:

[0798] The server organizes and integrates the collected data, automatically eliminating duplicate data and unnecessary information. For example, it aligns file information retrieved from cloud storage with calendar appointment information to create a single integrated data set.

[0799] Step 5:

[0800] The server then uses machine learning models to analyze the combined data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set.

[0801] Step 6:

[0802] The server sends the parsed personalized data to the device, formats the data in the specified format (e.g., JSON), and sends it to the device.

[0803] Step 7:

[0804] The device displays the received data through a user interface. The device displays personalized information (e.g., the latest schedule, important email notifications) on the application screen.

[0805] Step 8:

[0806] The user can review the provided data and take necessary actions. For example, the user can review the displayed schedule details, prepare for a meeting, or instantly reply to an important email.

[0807] Example 1

[0808] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0809] Conventional data collection and integration systems did not efficiently collect and organize data from different platforms, and duplicate and unnecessary data was not adequately eliminated. Furthermore, they lacked personalized functionality to enable users to quickly obtain the information they needed, reducing the efficiency of data management and information retrieval. This increased the burden of information management on users and led to problems with insufficient data utilization.

[0810] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0811] In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and personalizing it according to the user's needs; means for the terminal to send a data integration request to the server based on the user's operation; means for eliminating unnecessary and redundant information from the integrated data; and means for sending the generated personalized data to the terminal and displaying it on a user interface. This allows data from different platforms to be efficiently collected and organized, and redundant and unnecessary data to be eliminated, enabling users to quickly obtain the information they need. Furthermore, the personalization function reduces the user's burden of information management and improves data utilization.

[0812] "Different platforms" refers to different types of services and applications, such as cloud storage, calendar applications, and email services.

[0813] "API" stands for Application Programming Interface, an interface that allows software to communicate with other software.

[0814] "Means of collecting data" refers to the ability to obtain data through APIs of different platforms.

[0815] "Means for organizing and integrating data" refers to the function of organizing collected data, eliminating duplicate and unnecessary data, and generating a unified data set.

[0816] A "machine learning model" is an algorithm or model that analyzes data and personalizes information based on user behavior and needs.

[0817] "Means for personalization" refers to the function of customizing and providing information according to the needs and preferences of individual users.

[0818] The "means for the terminal to send a data integration request to the server based on a user's operation" is a function in which the user performs an operation through the device and sends a data integration request to the server based on that operation.

[0819] "Means for eliminating unnecessary and redundant information from integrated data" refers to a function that organizes and integrates collected data and eliminates redundant and unnecessary information.

[0820] "Means for transmitting the generated personalized data to the terminal and displaying it on the user interface" refers to a function for transmitting personalized information generated based on the analyzed data to the terminal and displaying it on the interface used by the user.

[0821] A "generative AI model" is an algorithm or model that uses artificial intelligence to perform generative tasks, particularly in data analysis and personalization.

[0822] A "prompt" is a natural language sentence input to a generative AI model to specify what the model should do.

[0823] The present invention relates to a data integration management system that collects data through APIs of different platforms and organizes and consolidates them. The system further analyzes the data using machine learning models and provides personalized information according to the user's needs. Specific embodiments of the present invention are described below.

[0824] System Configuration

[0825] The system mainly consists of the following components:

[0826] 1. Data collection server: Acquires data from different platforms via APIs, such as Google Calendar API, Gmail API, and cloud storage API.

[0827] 2. Data integration server: Organizes and integrates collected data, eliminating duplicate and unnecessary data.

[0828] 3. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models (e.g., TensorFlow, Scikit-learn) to generate personalized information based on the user's needs.

[0829] 4. User Interface: The interface through which users access personalized information. Examples include smartphone applications and web interfaces.

[0830] Program processing

[0831] Data collection

[0832] The server connects to the APIs of the various platforms used by the user and collects data. For example, if the user uses cloud storage, a calendar app, or an email service, the server collects data from each API (e.g., Google Calendar API, Gmail API, Dropbox API).

[0833] Data organization and integration

[0834] The server organizes and consolidates the data collected from each platform. During this process, duplicate data and unnecessary information are automatically eliminated. For example, if the same email exists on multiple devices, it will be consolidated into one. Duplicate calendar events will also be consolidated.

[0835] Data Analysis and Personalization

[0836] The server analyzes the integrated data using machine learning models (e.g., TensorFlow and Scikit-learn). The goal of the analysis is to understand the user's behavioral patterns and needs and personalize the data accordingly. For example, it can prioritize the most important meetings and tasks in a business person's schedule.

[0837] Providing data to users

[0838] The server then sends the generated personalized data to the device and provides it to the user through a user interface. For example, a notification may be displayed in a smartphone app, allowing the user to check it. Based on the user's request, the server also uses the generative AI model to extract and provide information related to the prompt sentence.

[0839] Specific examples

[0840] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0841] 2. The device sends requests to the server and collects data from different platforms.

[0842] 3. The server connects to the Google Calendar API, Gmail API, and Dropbox API to retrieve data from each service.

[0843] 4. The server analyzes the data and deletes and organizes duplicates. If there are emails with the same subject but different text, they are merged into one.

[0844] 5. The server uses machine learning models to analyze the data, extract key information relevant to the user's needs, and generate a personalized dataset.

[0845] 6. The device receives the data sent from the server and provides it to the user through a user interface. For example, important schedule or email notifications are displayed as app icons or on the screen.

[0846] 7. The user checks their schedule and important emails on their smartphone screen and takes appropriate action.

[0847] Prompt Sentence Examples

[0848] For example, by entering "What is my schedule for this week?", personalized schedule information will be displayed. By utilizing this prompt, users can obtain the information they need in natural language.

[0849] As a result, the present invention provides a system that efficiently collects and organizes data from different platforms and provides important information personalized based on user needs, thereby reducing the burden of information management on users and improving data utilization.

[0850] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0851] Step 1:

[0852] The user launches the app on their smartphone and taps the "Integrate Data" button.

[0853] Input: User action (tapping the "Integrate data" button)

[0854] Output: A data integration request is sent to the server

[0855] Step 2:

[0856] The terminal transmits a data collection request to the server based on the user's operation.

[0857] Input: Requests made by the user

[0858] Output: A data collection request is sent to the server

[0859] Step 3:

[0860] Based on the received request, the server accesses the APIs of various platforms to collect data, such as the Google Calendar API, Gmail API, and cloud storage API.

[0861] Input: Data Collection Request

[0862] Output: Data collected from each platform (e.g., calendar events, emails, files in storage)

[0863] Step 4:

[0864] The server then organizes and consolidates the collected data, a process that converts data from different formats into a common format and eliminates duplicate data and unnecessary information.

[0865] Input: Data collected from each platform

[0866] Output: Cleaned and consolidated dataset

[0867] Step 5:

[0868] The server inputs the integrated data into machine learning models (e.g., TensorFlow, Scikit-learn) and analyzes the data based on user behavior patterns and needs.

[0869] Input: Cleaned and consolidated dataset

[0870] Output: Analysis results (personalized information according to user needs)

[0871] Step 6:

[0872] The server uses the analysis to generate a personalized data set, including important emails for the user and reminders for upcoming meetings.

[0873] Input: Analysis results

[0874] Output: A personalized dataset

[0875] Step 7:

[0876] The server transmits the generated personalized data to the terminal and instructs the terminal to display the data on the user interface.

[0877] Input: personalized dataset

[0878] Output: Personalized information displayed in the user interface (e.g. smartphone notifications)

[0879] Step 8:

[0880] The user can then take the next action based on the personalized information displayed on the device, such as replying to an important email or preparing to attend a scheduled meeting.

[0881] Input: Personalization information displayed in the user interface

[0882] Output: User's next action (e.g., replying to an email, preparing for a meeting)

[0883] This is the specific processing flow of this system. This system enables users to manage information efficiently, greatly improving the utilization of data.

[0884] (Application example 1)

[0885] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0886] In modern society, it is extremely important to collect data from multiple platforms and devices and manage it centrally. However, integrating and organizing data from different platforms is not easy. Furthermore, advanced data analysis and personalization technologies are required to quickly provide users with the information they need from a wide range of information. Furthermore, it is also important to present personalized information in a visually easy-to-understand manner. The present invention has been proposed to solve these problems.

[0887] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0888] In this invention, the server includes means for collecting data through APIs of different platforms, means for organizing and integrating the collected data, and means for analyzing the data using a machine learning model and personalizing it according to the user's needs. This allows the data collected from different platforms to be organized and integrated, and the information required by the user to be personalized through analysis by the machine learning model, and to be visually provided via a display device.

[0889] "Distinct platforms" refers to multiple software and service offerings, each operated by a different vendor and using a different technology stack.

[0890] "API" stands for Application Program Interface, a set of rules and tools for exchanging data and functions between software programs.

[0891] "Means of collecting data" refers to the technology and devices used to obtain information from different platforms via APIs and aggregate it centrally.

[0892] "Data organization and integration methods" are techniques and processes that organize collected data in a consistent and coherent manner, eliminating duplicate and unnecessary information.

[0893] A "machine learning model" is a computer program that uses algorithms and statistical methods to analyze data and make predictions or recognize patterns.

[0894] "Personalization tools" are technologies and methods that customize collected and integrated data to meet the specific needs of users.

[0895] A "display device" is a device for presenting visual information to a user, such as smart glasses, a tablet, or a smartphone.

[0896] A "visual display" is a screen or projector for visually displaying digital information.

[0897] The "means of providing to the user" refers to the technology and protocols that appropriately transmit and present analyzed and personalized information to the device used by the user.

[0898] An embodiment of the present invention will be described.

[0899] The system of the present invention collects data through APIs of different platforms, organizes and integrates the collected data, analyzes it using machine learning models, and provides personalized information based on user needs using a visual display. The main components include a data collection server, a data integration server, an analysis server, and a display device such as smart glasses.

[0900] First, the server connects to the APIs of various platforms to collect user activity data, purchase history, etc. The collected data is sent to a data integration server, where it is organized and de-duplicated. The organized data is then sent to an analysis server, where it is analyzed using machine learning models. The analysis results are presented to the user as personalized information optimized for them. This information is presented via a visual display such as smart glasses.

[0901] Hardware and software used

[0902] Hardware:

[0903] Smart glasses (e.g. Google Glass, Vuzix Blade)

[0904] Data Collection Server

[0905] Data Integration Server

[0906] Analysis Server

[0907] Customer Relationship Management System (CRM)

[0908] software:

[0909] API connection libraries (e.g., Postman, Retrofit)

[0910] Machine learning libraries (e.g. TensorFlow, scikit-learn)

[0911] Database (e.g. MySQL, MongoDB)

[0912] UI development frameworks (e.g., Angular, React)

[0913] Processing flow and example

[0914] First, when a user visits the store, the server collects data via APIs from different platforms, such as cloud storage, calendar services, and email services. For example, information about the user's past purchase history and browsing history is obtained. The collected data is then organized and integrated by a data integration server. During this process, duplicate and unnecessary data is eliminated, creating a consistent data set.

[0915] The analytics server then uses machine learning models based on this integrated data to analyze the information needed to meet the user's needs and generate personalized information, such as recommendations for new camping equipment based on the user's previous purchases, or new product recommendations.

[0916] Finally, the generated personalized information is provided to the user through a visual display such as smart glasses, allowing store clerks to make more accurate product recommendations to customers, which is expected to improve customer satisfaction.

[0917] Specific prompt examples

[0918] Provide information to the machine learning model using prompt statements such as:

[0919] Prompt statement:

[0920] "Please recommend some products to a male customer in his 30s who enjoys the outdoors and has purchased a lot of camping equipment in the past year."

[0921] Example response:

[0922] "I would like to recommend a new camping stove and waterproof tent to this customer. I can also offer a great deal on our recently arrived multi-purpose knife."

[0923] In this way, the system of the present invention can efficiently collect and integrate data from different platforms, use machine learning models to personalize information based on the user's needs, and provide it through a visual display, thereby improving the user experience.

[0924] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0925] Step 1:

[0926] The server connects to the APIs of different platforms (e.g., cloud storage, calendar services, email services) to collect user activity data, purchase history, etc. Specifically, it sends an HTTP request to each API endpoint and retrieves the returned data. The input to this step is the API endpoint of each platform and user authentication information, and the output is raw data of the user's activity data and purchase history.

[0927] Step 2:

[0928] The server sends the collected data to the data integration server, which organizes and integrates the data. Specifically, it standardizes the data format and eliminates duplicate data and unnecessary information. This process involves merging, for example, duplicate purchase histories or identical event information into one. The input to this step is the raw data collected from each platform, and the output is an organized and integrated dataset.

[0929] Step 3:

[0930] The server sends the consolidated data to the analysis server, where it is analyzed using a machine learning model. Specifically, data analysis is performed to predict the user's interests and preferences based on past purchase data and activity history. This process recommends products and services that best suit the user's needs. The input to this step is the consolidated data set, and the output is personalized recommendations for the user.

[0931] Step 4:

[0932] The server then transmits the analyzed personalized information to a display device (e.g., smart glasses) to provide it to the user. Specifically, the server converts the generated recommendation information into a format suitable for display on the device screen and transmits it to the smart glasses. The input of this step is the personalized information generated by the machine learning model, and the output is the recommendation information displayed on the visual display.

[0933] Step 5:

[0934] The user checks the personalized information presented through the smart glasses and takes appropriate action. Specifically, the system explains recommended products and services to the customer and helps promote purchases. The input of this step is the recommended information displayed on the smart glasses, and the output is the user's action (e.g., product suggestions or purchasing procedures).

[0935] Through each of the above steps, data collected from different platforms is efficiently integrated, and personalized information is provided to users based on the results of analysis using machine learning models, thereby achieving advanced customer service and improved service.

[0936] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0937] This invention relates to a data integration management system that collects, organizes, and integrates data through APIs from different platforms. Furthermore, it analyzes the data using machine learning models and provides personalized information according to the user's needs and emotions. By combining it with an emotion engine that recognizes the user's emotions, even more advanced personalization becomes possible.

[0938] System Configuration

[0939] The system mainly consists of the following components:

[0940] 1. Data collection server: Acquires data from different platforms via API.

[0941] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[0942] 3. Emotion engine: Recognizes user emotions and reflects them in data analysis.

[0943] 4. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models to generate personalized information based on the user's needs and emotions.

[0944] 5. User Interface: The interface through which the user accesses personalized information.

[0945] Program processing

[0946] Data collection

[0947] The server connects to the APIs of the various platforms the user uses and collects data. For example, if the user uses cloud storage, a calendar app, an email service, or a social networking service, the server collects data from each API.

[0948] Data organization and integration

[0949] The server organizes and integrates the data collected from each platform. In doing so, it automatically eliminates duplicate data and unnecessary information. For example, it aligns posts obtained from social media with communications from email services and provides them as an integrated data set.

[0950] emotion recognition

[0951] The emotion engine recognizes the user's emotions. It analyzes user input, voice data, facial expression data, etc. to identify the emotion the user is currently feeling. This emotion data is reflected in subsequent data analysis and personalization.

[0952] Data Analysis and Personalization

[0953] The server then analyzes the integrated data using machine learning models to extract the most relevant information based on the user's past behavior, current situation, and perceived emotions, generating a personalized data set. For example, if the user is feeling stressed, it will prioritize relaxation information and entertainment content.

[0954] Providing data to users

[0955] The device provides the analyzed personalized data to the user, helping the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing users to check important schedules, messages, and emotional content.

[0956] Specific examples

[0957] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[0958] 2. The device sends a request to the server to collect data from different platforms.

[0959] 3. The server collects data from cloud storage, calendars, email services, and social media.

[0960] 4. The server consolidates and organizes the collected data to create a consistent data set.

[0961] 5. The emotion engine recognizes the user's emotions and provides the data to the server.

[0962] 6. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs and perceived emotions.

[0963] 7. The device provides personalized information to the user through a user interface.

[0964] 8. Users can view schedules, important emails, and emotional suggestions (such as entertainment content or relaxation methods) generated on their smartphone screen, and take action to improve productivity.

[0965] As described above, the system of the present invention efficiently integrates data from a variety of devices and cloud services, providing an environment in which users can quickly access the information they need. In particular, by combining it with an emotion engine, advanced personalization based on the user's emotional state can be achieved, making it possible to better meet individual needs. This system is expected to significantly reduce the data management burden on users, particularly business people, and improve the quality of their daily lives.

[0966] The processing flow will be explained below.

[0967] Step 1:

[0968] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[0969] Step 2:

[0970] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[0971] Step 3:

[0972] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services, SNS) to collect data. The server sends an HTTP request to the service's API endpoint and temporarily stores the collected data in memory.

[0973] Step 4:

[0974] The server organizes and integrates the data it collects. For example, it combines file data collected from cloud storage and schedule data collected from a calendar app into a single integrated data set. It also eliminates duplicate and unnecessary data.

[0975] Step 5:

[0976] The emotion engine recognizes the user's emotions. The user's voice data, text data, or image data obtained from the terminal or external device is sent to the server, which inputs it into the emotion engine. The emotion engine analyzes this data and recognizes the user's emotional state (e.g., joy, sadness, stress).

[0977] Step 6:

[0978] The server uses machine learning models to analyze the integrated data. The server analyzes the data based on the user's past behavioral history, current situation, and recognized emotions, and extracts the information most appropriate for the user. For example, if the user is feeling stressed, it will prioritize information related to relaxation.

[0979] Step 7:

[0980] The server sends the parsed personalized data to the device. The server formats the data in the specified format (e.g., JSON) and sends it to the device. The data includes important schedules, emails, SNS notifications, etc.

[0981] Step 8:

[0982] The device displays the received data through a user interface, and the device displays personalized information (e.g., the latest schedule, important email notifications, and emotional entertainment content) on the application screen.

[0983] Step 9:

[0984] The user can review the provided data and take necessary actions. For example, the user can check the displayed schedule details and prepare for a meeting. The user can also reduce stress by viewing relaxing content tailored to the user's emotions.

[0985] As described above, this system collects and integrates data from the user's various devices and cloud services, and by using an emotion engine, achieves advanced personalization according to the user's emotional state, allowing users to quickly access the information they need most and supporting efficient daily life and work performance.

[0986] Example 2

[0987] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0988] Currently, collecting, integrating, and analyzing data from different sources is a time-consuming and labor-intensive task. It is also difficult to provide personalized information that takes into account the user's emotions, making it difficult to provide appropriate services that meet the user's needs. Furthermore, issues such as data duplication and organization are not taken into consideration, resulting in a lack of user convenience.

[0989] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data through application programming interfaces of different information sources, means for organizing and integrating the collected data, means for eliminating duplicate data, means for using an emotion engine to recognize the user's emotions, means for analyzing the data using a machine learning algorithm and personalizing it according to the user's needs and emotions, and means for providing the personalized data to the user. This makes it possible to efficiently collect and integrate data from multiple information sources and provide personalized information that takes the user's emotions into consideration.

[0990] "Sources" refer to the systems and platforms that provide data, including cloud storage, calendar apps, email services, and social networking services.

[0991] An "application programming interface" is a defined set of communication rules that allow software applications to interact with other applications and services.

[0992] "Data collection" refers to the process of obtaining data from different sources and storing it on a server.

[0993] Data consolidation is the process of organizing collected data into a consistent format, linking it where necessary, and eliminating duplicate data.

[0994] "Duplicate data elimination" is the process of detecting and removing duplicate data to prevent the same information from being collected multiple times.

[0995] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions, text, voice, etc. to recognize emotions.

[0996] A "machine learning algorithm" is a mathematical model or method for learning patterns from large amounts of data and making predictions or classifications.

[0997] "Personalization" is the process of providing information optimized to the needs and emotions of individual users based on collected data and analysis results.

[0998] "Means for providing to the user" refers to an interface or notification system for providing the analyzed information in a form that the user can use.

[0999] This invention is a system that collects data through application programming interfaces (APIs) from different information sources, organizes, integrates, and analyzes them, and provides personalized information according to the user's needs and emotions. The main components of the system include the following:

[1000] 1. Data collection server: The server connects to the APIs of cloud storage, calendar apps, email services, social networking sites, etc. used by users to collect data. Specifically, it uses APIs such as Google Drive, Google Calendar, Gmail, and Twitter.

[1001] 2. Data integration server: The server integrates the collected data and organizes it into a consistent format. This process also eliminates duplicate data. Specifically, the server processes and formats the data using the Python libraries Pandas and NumPy.

[1002] 3. Emotion Engine: The emotion engine is designed to recognize the user's emotions. The server uses an emotion recognition API (e.g., Emotion API) to analyze the text data, voice data, and facial expression data entered by the user.

[1003] 4. Analytics Server: The server analyzes the data using machine learning algorithms to generate personalized information based on the user's past behavior, current situation, and perceived emotions. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch.

[1004] 5. User Interface: The device is responsible for providing personalized information to the user. It displays the information as notifications and messages through smartphone applications and web interfaces, making it easily accessible to the user.

[1005] For example, when a user launches a dedicated smartphone app and taps the "Integrate Data" button, the device sends a data collection request to the server. The server accesses various APIs to collect, organize, and integrate the necessary data. The emotion engine then analyzes the text and facial expression data to identify the user's emotions. The data is analyzed using machine learning algorithms to generate personalized information, which the device then provides to the user as notifications or messages. This series of processes allows users to quickly access the information they need and reduces the burden of information management.

[1006] An example prompt is:

[1007] "I want to develop a system that integrates and organizes data from different platforms, recognizes user emotions, and provides analytical data. I will use EmotionAPI for emotion recognition and TensorFlow for machine learning. Please explain the program process for this system step by step."

[1008] This system is expected to improve the quality of daily life and business by enabling users to integrate data from a variety of sources and obtain the most appropriate information based on their emotions.

[1009] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1010] Step 1:

[1011] Data collection

[1012] The server connects to the application programming interfaces (APIs) of various information sources used by the user and collects data. It receives authentication from each information source using an API key or OAuth token as input. For example, it retrieves the user's schedule data from the Google Calendar API and email data from the Gmail API. As output, it stores the collected data in JSON format on the server.

[1013] Specifically, the server sends the following API request:

[1014] GET https: / / www.googleapis.com / calendar / v3 / calendars / primary / events?key=API_KEY

[1015] This request returns the event data from Google Calendar.

[1016] Step 2:

[1017] Data organization and integration

[1018] The server organizes and consolidates the collected data. It uses data from each source stored in JSON format as input. It standardizes the data format, deletes unnecessary data, and eliminates duplicate data. Specifically, it uses Python's Pandas and NumPy libraries to handle the data as a data frame, extracts and formats the necessary information, and generates a consistent dataset as output.

[1019] Specifically, the server performs the following operations:

[1020] import pandas as pd

[1021] calendar_data = pd.read_json('calendar_data.json')

[1022] email_data = pd.read_json('email_data.json')

[1023] merged_data = pd.merge(calendar_data, email_data, on='event_id', how='outer')

[1024] This will merge your calendar and email data.

[1025] Step 3:

[1026] emotion recognition

[1027] The emotion engine recognizes the user's emotions. As input, it uses the user's text data, voice data, and facial expression data. It sends the data to an emotion recognition API to perform emotion analysis. For example, it uses Microsoft's Emotion API to obtain the emotional state from the text data. As output, an emotion score is returned to the server.

[1028] Specifically, the server sends the following API request:

[1029] POST https: / / westus.api.cognitive.microsoft.com / text / analytics / v3.0 / sentiment

[1030] This request returns a sentiment score for the text data.

[1031] Step 4:

[1032] Data Analysis and Personalization

[1033] The server uses machine learning algorithms to analyze the data and generate personalized information. It uses the cleaned and consolidated data and sentiment scores as input. It applies predictive models based on historical data and sentiment data using TensorFlow or PyTorch. As output, it generates personalized information tailored to the user's needs.

[1034] Specifically, the server performs the following operations:

[1035] import tensorflow as tf

[1036] model = tf.keras.models.load_model('model.h5')

[1037] predictions = model.predict(merged_data)

[1038] This operation generates personalized information.

[1039] Step 5:

[1040] Providing data to users

[1041] The device provides personalized information through a user interface. It uses the parsed personalized information as input, provides it to the user as notifications or messages through a smartphone application or web interface, and displays the information the user needs as output.

[1042] Specifically, the device performs the following operations:

[1043] NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);

[1044] Notification notification = new Notification.Builder(this)

[1045] .setContentTitle("Relaxation Guide")

[1046] .setContentText("Click here for a 10-minute guided meditation to help you relax right now")

[1047] .setSmallIcon(R.drawable.ic_notification)

[1048] .build();

[1049] notificationManager.notify(notificationId, notification);

[1050] This action will cause a notification to appear on the user's device.

[1051] Through the above steps, a system is realized that allows users to integrate data from a variety of information sources and provide optimal information according to their emotions.

[1052] (Application example 2)

[1053] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1054] In industrial environments, it is necessary to collect and integrate data from multiple different platforms and systems to obtain accurate information. However, there is a lack of mechanisms for efficiently collecting and integrating individual data, resulting in issues such as data duplication and the cumbersome task of organizing it. Furthermore, there is a lack of personalized support based on the emotions of workers, which prevents sufficient improvements in production efficiency and safety.

[1055] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and providing personalized information according to the user's needs and emotions; means for generating suggestions for improving efficiency and safety in the work environment based on the integrated data; means for providing the suggestions to the user through a user interface and providing feedback in real time; and means for recognizing the user's emotions and reflecting the emotional information in the analyzed data. This makes it possible to efficiently collect data from multiple data sources and support real-time improvements in the efficiency and safety of industrial environments based on the integrated information.

[1056] A "platform API" is an interface for exchanging and manipulating data between different systems and devices.

[1057] "Means of collecting data" refers to a mechanism for obtaining the necessary information through APIs of different platforms.

[1058] "Means for organizing and integrating data" refers to a mechanism for systematically organizing collected information, eliminating duplicate data, and consolidating it into a single data set.

[1059] A "machine learning model" is an algorithm for analyzing large amounts of data and finding patterns, and is a model for making adaptive predictions and suggestions based on the results obtained from the analysis.

[1060] "Personalized information" is information that is customized according to the individual needs and feelings of the user.

[1061] The "means for generating proposals" is a mechanism for creating specific actions and proposals for users based on the analyzed data.

[1062] A "user interface" is a screen or operating means that allows a user to exchange information with a system.

[1063] "Means of providing feedback in real time" refers to a mechanism that instantly notifies the user of analysis results and suggestions, and responds immediately to the user's actions and choices.

[1064] A "means for recognizing emotions" is a mechanism for analyzing emotions from the user's facial expressions, voice, etc., and converting those emotions into data.

[1065] The present invention relates to a data integration management system for improving efficiency and safety in an industrial environment. The system is implemented using the following hardware and software.

[1066] System Configuration

[1067] The system includes the following components:

[1068] 1. Data collection server: Connects to APIs of different platforms to collect data. Specific API examples include factory sensor device APIs and surveillance camera APIs.

[1069] 2. Data integration server: Organizes the collected data, eliminates duplicates, and integrates them. A database such as MongoDB is used to generate an integrated data set.

[1070] 3. Emotion recognition engine: Recognizes emotions from the user's facial expressions and voice, and generates emotional data. Specifically, it uses the Emotion API.

[1071] 4. Analysis Server: Analyzes the integrated data using machine learning models (e.g., built using TensorFlow or PyTorch) and generates personalized information according to the user's needs and emotions.

[1072] 5. User Interface: Provide feedback to the robot using the Robot Operating System (ROS).

[1073] Processing Description

[1074] The data collection server collects data via API from factory sensor devices, surveillance cameras, inventory systems, etc. This allows the real-time data obtained from each platform to be integrated.

[1075] The data integration server organizes the collected data, eliminating duplicates and generating a consistent dataset, which is stored in MongoDB for efficient data manipulation.

[1076] The emotion recognition engine analyzes the user's facial expressions and voice data acquired from a camera or microphone to generate emotion data. This emotion data is added to the integrated data and reflected in the analysis.

[1077] The analytics server runs machine learning models using the integrated data and sentiment data to generate recommendations to improve production efficiency and safety, including real-time work efficiency improvements, break recommendations, and inventory reordering.

[1078] The user interface uses ROS to provide specific feedback, such as prompting workers to take a break or showing instructions on how to avoid hazards.

[1079] Specific examples

[1080] 1. While a user is working in a factory, a camera captures the user's facial expression data.

[1081] 2. The data collection server collects sensor data and surveillance camera video data via API.

[1082] 3. The data integration server organizes this data and generates a consistent dataset.

[1083] 4. The emotion recognition engine recognizes the user's emotions and adds them to the integrated data.

[1084] 5. The analytics server analyzes the data using machine learning models to generate personalized recommendations.

[1085] 6. The user interface provides real-time feedback to the user through ROS.

[1086] Prompt Sentence Examples

[1087] Integrate factory sensor data, surveillance camera footage, inventory data, and emotion data obtained from workers' facial expressions, analyze the data using machine learning models, and generate suggestions to improve production efficiency and work safety in the factory. Specifically, provide feedback including actions such as suggesting a break if a worker is tired, or automating reordering if inventory is low.

[1088] This system can efficiently collect data from multiple sources and use the analysis results to help improve the efficiency and safety of industrial environments in real time, which is expected to improve factory productivity and worker satisfaction.

[1089] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1090] Step 1:

[1091] Data collection

[1092] The server collects data from APIs such as factory sensor devices, surveillance cameras, and inventory systems. Although the data format obtained from each device is different, it accesses each API endpoint to obtain sensor information, video data, and inventory information. The input to this step is the raw data obtained from the API, and the output is a temporarily stored dataset. Specifically, it sends an HTTP request to the API endpoint and obtains the response data.

[1093] Step 2:

[1094] Data Integration

[1095] The server integrates the collected data in different formats and eliminates duplicates. To do this, it organizes the data using a database such as MongoDB. The input to this step is the collected raw data, and the output is an integrated dataset. Specifically, it reads the data stored in the database, matches the data structure, and performs a process to remove duplicate items.

[1096] Step 3:

[1097] emotion recognition

[1098] The server inputs image data acquired from the surveillance camera into an emotion recognition engine to analyze the user's emotions. This engine uses the Emotion API to analyze emotions from the image data and output it as data. The input for this step is image data, and the output is analyzed emotion data. Specifically, the image data is sent to the Emotion API and emotion-tagged data is received.

[1099] Step 4:

[1100] Data analysis

[1101] The server analyzes the integrated data and sentiment data using machine learning models such as TensorFlow and PyTorch. This analysis generates personalized suggestions based on the user's needs and sentiment. The inputs to this step are the integrated dataset and sentiment data, and the output is personalized suggestions. Specifically, the data is preprocessed, input to the machine learning model, and the analysis results are obtained.

[1102] Step 5:

[1103] Proposal Generation

[1104] The server generates proposals to improve the factory's production efficiency and safety based on the results of the data analysis. The input to this step is the analysis results, and the output is specific proposals. Specific operations include formatting the proposals based on the analysis results and generating actions to improve work efficiency, propose breaks, and reorder inventory.

[1105] Step 6:

[1106] Providing feedback

[1107] The server provides real-time feedback to the user through a user interface. Specifically, it uses the Robot Operating System (ROS) to notify the worker and prompt them to take specific actions. The input of this step is the generated suggestion, and the output is the feedback displayed to the user. The specific action is to communicate the suggestion to the user through ROS.

[1108] Step 7:

[1109] Monitoring User Actions

[1110] The device monitors how the user responds to the feedback. This accumulates data that can be used for future analysis. The input for this step is the user's behavioral data, and the output is saved as reaction data. Specifically, the device collects the user's behavioral log and saves it in a database that will be used for the next data analysis.

[1111] As a result, it is possible to efficiently collect information from multiple data sources and, based on the analysis results, to help improve the efficiency and safety of industrial environments in real time.

[1112] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1113] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1114] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1115] [Fourth embodiment]

[1116] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1117] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1118] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1119] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1120] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1121] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1122] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1123] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1124] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1125] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1126] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1127] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1128] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1129] This invention relates to a data integration management system that collects, organizes, and consolidates data through APIs from different platforms, and further analyzes the data using machine learning models to provide personalized information according to user needs.

[1130] System Configuration

[1131] The system mainly consists of the following components:

[1132] 1. Data collection server: Acquires data from different platforms via API.

[1133] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[1134] 3. Analysis Server: Based on the integrated data, it uses machine learning models to analyze the data and generate personalized information based on the user's needs.

[1135] 4. User Interface: The interface through which the user accesses personalized information.

[1136] Program processing

[1137] Data collection

[1138] The server connects to the APIs of the various platforms used by the user and collects data. For example, if a user uses cloud storage, a calendar app, or an email service, the server collects data from each API.

[1139] Data organization and integration

[1140] The server then organizes and consolidates the data collected from each platform, automatically eliminating duplicate data and unnecessary information. For example, calendar events and email content are consolidated and compiled into a single data set for each user.

[1141] Data Analysis and Personalization

[1142] The server then uses machine learning models to analyze the integrated data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set. For example, a businessperson might prioritize schedule changes and important emails.

[1143] Providing data to users

[1144] The device provides the analyzed personalized data to the user, and helps the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing the user to check important appointments and messages.

[1145] Specific examples

[1146] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[1147] 2. The device sends requests to the server and collects data from different platforms.

[1148] 3. The server collects data from cloud storage, calendar, and email services through APIs.

[1149] 4. The server organizes and consolidates the collected data to create a coherent data set.

[1150] 5. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs.

[1151] 6. The device provides personalized information to the user through a user interface.

[1152] 7. Users can view schedules and important emails generated on their smartphone screens and take action to improve their productivity.

[1153] As described above, the system of the present invention efficiently integrates data from various devices and cloud services, providing an environment in which users can quickly access the information they need. This system is expected to significantly reduce the data management burden on users, especially business people.

[1154] The processing flow will be explained below.

[1155] Step 1:

[1156] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[1157] Step 2:

[1158] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[1159] Step 3:

[1160] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services) to collect data. The server sends an HTTP request to the service's API endpoint and stores the collected data in memory.

[1161] Step 4:

[1162] The server organizes and integrates the collected data, automatically eliminating duplicate data and unnecessary information. For example, it aligns file information retrieved from cloud storage with calendar appointment information to create a single integrated data set.

[1163] Step 5:

[1164] The server then uses machine learning models to analyze the combined data, extracting the most relevant information based on the user's past behavior and current situation, and generating a personalized data set.

[1165] Step 6:

[1166] The server sends the parsed personalized data to the device, formats the data in the specified format (e.g., JSON), and sends it to the device.

[1167] Step 7:

[1168] The device displays the received data through a user interface. The device displays personalized information (e.g., the latest schedule, important email notifications) on the application screen.

[1169] Step 8:

[1170] The user can review the provided data and take necessary actions. For example, the user can review the displayed schedule details, prepare for a meeting, or instantly reply to an important email.

[1171] Example 1

[1172] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1173] Conventional data collection and integration systems did not efficiently collect and organize data from different platforms, and duplicate and unnecessary data was not adequately eliminated. Furthermore, they lacked personalized functionality to enable users to quickly obtain the information they needed, reducing the efficiency of data management and information retrieval. This increased the burden of information management on users and led to problems with insufficient data utilization.

[1174] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1175] In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and personalizing it according to the user's needs; means for the terminal to send a data integration request to the server based on the user's operation; means for eliminating unnecessary and redundant information from the integrated data; and means for sending the generated personalized data to the terminal and displaying it on a user interface. This allows data from different platforms to be efficiently collected and organized, and redundant and unnecessary data to be eliminated, enabling users to quickly obtain the information they need. Furthermore, the personalization function reduces the user's burden of information management and improves data utilization.

[1176] "Different platforms" refers to different types of services and applications, such as cloud storage, calendar applications, and email services.

[1177] "API" stands for Application Programming Interface, an interface that allows software to communicate with other software.

[1178] "Means of collecting data" refers to the ability to obtain data through APIs of different platforms.

[1179] "Means for organizing and integrating data" refers to the function of organizing collected data, eliminating duplicate and unnecessary data, and generating a unified data set.

[1180] A "machine learning model" is an algorithm or model that analyzes data and personalizes information based on user behavior and needs.

[1181] "Means for personalization" refers to the function of customizing and providing information according to the needs and preferences of individual users.

[1182] The "means for the terminal to send a data integration request to the server based on a user's operation" is a function in which the user performs an operation through the device and sends a data integration request to the server based on that operation.

[1183] "Means for eliminating unnecessary and redundant information from integrated data" refers to a function that organizes and integrates collected data and eliminates redundant and unnecessary information.

[1184] "Means for transmitting the generated personalized data to the terminal and displaying it on the user interface" refers to a function for transmitting personalized information generated based on the analyzed data to the terminal and displaying it on the interface used by the user.

[1185] A "generative AI model" is an algorithm or model that uses artificial intelligence to perform generative tasks, particularly in data analysis and personalization.

[1186] A "prompt" is a natural language sentence input to a generative AI model to specify what the model should do.

[1187] The present invention relates to a data integration management system that collects data through APIs of different platforms and organizes and consolidates them. The system further analyzes the data using machine learning models and provides personalized information according to the user's needs. Specific embodiments of the present invention are described below.

[1188] System Configuration

[1189] The system mainly consists of the following components:

[1190] 1. Data collection server: Acquires data from different platforms via APIs, such as Google Calendar API, Gmail API, and cloud storage API.

[1191] 2. Data integration server: Organizes and integrates collected data, eliminating duplicate and unnecessary data.

[1192] 3. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models (e.g., TensorFlow, Scikit-learn) to generate personalized information based on the user's needs.

[1193] 4. User Interface: The interface through which users access personalized information. Examples include smartphone applications and web interfaces.

[1194] Program processing

[1195] Data collection

[1196] The server connects to the APIs of the various platforms used by the user and collects data. For example, if the user uses cloud storage, a calendar app, or an email service, the server collects data from each API (e.g., Google Calendar API, Gmail API, Dropbox API).

[1197] Data organization and integration

[1198] The server organizes and consolidates the data collected from each platform. During this process, duplicate data and unnecessary information are automatically eliminated. For example, if the same email exists on multiple devices, it will be consolidated into one. Duplicate calendar events will also be consolidated.

[1199] Data Analysis and Personalization

[1200] The server analyzes the integrated data using machine learning models (e.g., TensorFlow and Scikit-learn). The goal of the analysis is to understand the user's behavioral patterns and needs and personalize the data accordingly. For example, it can prioritize the most important meetings and tasks in a business person's schedule.

[1201] Providing data to users

[1202] The server then sends the generated personalized data to the device and provides it to the user through a user interface. For example, a notification may be displayed in a smartphone app, allowing the user to check it. Based on the user's request, the server also uses the generative AI model to extract and provide information related to the prompt sentence.

[1203] Specific examples

[1204] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[1205] 2. The device sends requests to the server and collects data from different platforms.

[1206] 3. The server connects to the Google Calendar API, Gmail API, and Dropbox API to retrieve data from each service.

[1207] 4. The server analyzes the data and deletes and organizes duplicates. If there are emails with the same subject but different text, they are merged into one.

[1208] 5. The server uses machine learning models to analyze the data, extract key information relevant to the user's needs, and generate a personalized dataset.

[1209] 6. The device receives the data sent from the server and provides it to the user through a user interface. For example, important schedule or email notifications are displayed as app icons or on the screen.

[1210] 7. The user checks their schedule and important emails on their smartphone screen and takes appropriate action.

[1211] Prompt Sentence Examples

[1212] For example, by entering "What is my schedule for this week?", personalized schedule information will be displayed. By utilizing this prompt, users can obtain the information they need in natural language.

[1213] As a result, the present invention provides a system that efficiently collects and organizes data from different platforms and provides important information personalized based on user needs, thereby reducing the burden of information management on users and improving data utilization.

[1214] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1215] Step 1:

[1216] The user launches the app on their smartphone and taps the "Integrate Data" button.

[1217] Input: User action (tapping the "Integrate data" button)

[1218] Output: A data integration request is sent to the server

[1219] Step 2:

[1220] The terminal transmits a data collection request to the server based on the user's operation.

[1221] Input: Requests made by the user

[1222] Output: A data collection request is sent to the server

[1223] Step 3:

[1224] Based on the received request, the server accesses the APIs of various platforms to collect data, such as the Google Calendar API, Gmail API, and cloud storage API.

[1225] Input: Data Collection Request

[1226] Output: Data collected from each platform (e.g., calendar events, emails, files in storage)

[1227] Step 4:

[1228] The server then organizes and consolidates the collected data, a process that converts data from different formats into a common format and eliminates duplicate data and unnecessary information.

[1229] Input: Data collected from each platform

[1230] Output: Cleaned and consolidated dataset

[1231] Step 5:

[1232] The server inputs the integrated data into machine learning models (e.g., TensorFlow, Scikit-learn) and analyzes the data based on user behavior patterns and needs.

[1233] Input: Cleaned and consolidated dataset

[1234] Output: Analysis results (personalized information according to user needs)

[1235] Step 6:

[1236] The server uses the analysis to generate a personalized data set, including important emails for the user and reminders for upcoming meetings.

[1237] Input: Analysis results

[1238] Output: A personalized dataset

[1239] Step 7:

[1240] The server transmits the generated personalized data to the terminal and instructs the terminal to display the data on the user interface.

[1241] Input: personalized dataset

[1242] Output: Personalized information displayed in the user interface (e.g. smartphone notifications)

[1243] Step 8:

[1244] The user can then take the next action based on the personalized information displayed on the device, such as replying to an important email or preparing to attend a scheduled meeting.

[1245] Input: Personalization information displayed in the user interface

[1246] Output: User's next action (e.g., replying to an email, preparing for a meeting)

[1247] This is the specific processing flow of this system. This system enables users to manage information efficiently, greatly improving the utilization of data.

[1248] (Application example 1)

[1249] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1250] In modern society, it is extremely important to collect data from multiple platforms and devices and manage it centrally. However, integrating and organizing data from different platforms is not easy. Furthermore, advanced data analysis and personalization technologies are required to quickly provide users with the information they need from a wide range of information. Furthermore, it is also important to present personalized information in a visually easy-to-understand manner. The present invention has been proposed to solve these problems.

[1251] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1252] In this invention, the server includes means for collecting data through APIs of different platforms, means for organizing and integrating the collected data, and means for analyzing the data using a machine learning model and personalizing it according to the user's needs. This allows the data collected from different platforms to be organized and integrated, and the information required by the user to be personalized through analysis by the machine learning model, and to be visually provided via a display device.

[1253] "Distinct platforms" refers to multiple software and service offerings, each operated by a different vendor and using a different technology stack.

[1254] "API" stands for Application Program Interface, a set of rules and tools for exchanging data and functions between software programs.

[1255] "Means of collecting data" refers to the technology and devices used to obtain information from different platforms via APIs and aggregate it centrally.

[1256] "Data organization and integration methods" are techniques and processes that organize collected data in a consistent and coherent manner, eliminating duplicate and unnecessary information.

[1257] A "machine learning model" is a computer program that uses algorithms and statistical methods to analyze data and make predictions or recognize patterns.

[1258] "Personalization tools" are technologies and methods that customize collected and integrated data to meet the specific needs of users.

[1259] A "display device" is a device for presenting visual information to a user, such as smart glasses, a tablet, or a smartphone.

[1260] A "visual display" is a screen or projector for visually displaying digital information.

[1261] The "means of providing to the user" refers to the technology and protocols that appropriately transmit and present analyzed and personalized information to the device used by the user.

[1262] An embodiment of the present invention will be described.

[1263] The system of the present invention collects data through APIs of different platforms, organizes and integrates the collected data, analyzes it using machine learning models, and provides personalized information based on user needs using a visual display. The main components include a data collection server, a data integration server, an analysis server, and a display device such as smart glasses.

[1264] First, the server connects to the APIs of various platforms to collect user activity data, purchase history, etc. The collected data is sent to a data integration server, where it is organized and de-duplicated. The organized data is then sent to an analysis server, where it is analyzed using machine learning models. The analysis results are presented to the user as personalized information optimized for them. This information is presented via a visual display such as smart glasses.

[1265] Hardware and software used

[1266] Hardware:

[1267] Smart glasses (e.g. Google Glass, Vuzix Blade)

[1268] Data Collection Server

[1269] Data Integration Server

[1270] Analysis Server

[1271] Customer Relationship Management System (CRM)

[1272] software:

[1273] API connection libraries (e.g., Postman, Retrofit)

[1274] Machine learning libraries (e.g. TensorFlow, scikit-learn)

[1275] Database (e.g. MySQL, MongoDB)

[1276] UI development frameworks (e.g., Angular, React)

[1277] Processing flow and example

[1278] First, when a user visits the store, the server collects data via APIs from different platforms, such as cloud storage, calendar services, and email services. For example, information about the user's past purchase history and browsing history is obtained. The collected data is then organized and integrated by a data integration server. During this process, duplicate and unnecessary data is eliminated, creating a consistent data set.

[1279] The analytics server then uses machine learning models based on this integrated data to analyze the information needed to meet the user's needs and generate personalized information, such as recommendations for new camping equipment based on the user's previous purchases, or new product recommendations.

[1280] Finally, the generated personalized information is provided to the user through a visual display such as smart glasses, allowing store clerks to make more accurate product recommendations to customers, which is expected to improve customer satisfaction.

[1281] Specific prompt examples

[1282] Provide information to the machine learning model using prompt statements such as:

[1283] Prompt statement:

[1284] "Please recommend some products to a male customer in his 30s who enjoys the outdoors and has purchased a lot of camping equipment in the past year."

[1285] Example response:

[1286] "I would like to recommend a new camping stove and waterproof tent to this customer. I can also offer a great deal on our recently arrived multi-purpose knife."

[1287] In this way, the system of the present invention can efficiently collect and integrate data from different platforms, use machine learning models to personalize information based on the user's needs, and provide it through a visual display, thereby improving the user experience.

[1288] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1289] Step 1:

[1290] The server connects to the APIs of different platforms (e.g., cloud storage, calendar services, email services) to collect user activity data, purchase history, etc. Specifically, it sends an HTTP request to each API endpoint and retrieves the returned data. The input to this step is the API endpoint of each platform and user authentication information, and the output is raw data of the user's activity data and purchase history.

[1291] Step 2:

[1292] The server sends the collected data to the data integration server, which organizes and integrates the data. Specifically, it standardizes the data format and eliminates duplicate data and unnecessary information. This process involves merging, for example, duplicate purchase histories or identical event information into one. The input to this step is the raw data collected from each platform, and the output is an organized and integrated dataset.

[1293] Step 3:

[1294] The server sends the consolidated data to the analysis server, where it is analyzed using a machine learning model. Specifically, data analysis is performed to predict the user's interests and preferences based on past purchase data and activity history. This process recommends products and services that best suit the user's needs. The input to this step is the consolidated data set, and the output is personalized recommendations for the user.

[1295] Step 4:

[1296] The server then transmits the analyzed personalized information to a display device (e.g., smart glasses) to provide it to the user. Specifically, the server converts the generated recommendation information into a format suitable for display on the device screen and transmits it to the smart glasses. The input of this step is the personalized information generated by the machine learning model, and the output is the recommendation information displayed on the visual display.

[1297] Step 5:

[1298] The user checks the personalized information presented through the smart glasses and takes appropriate action. Specifically, the system explains recommended products and services to the customer and helps promote purchases. The input of this step is the recommended information displayed on the smart glasses, and the output is the user's action (e.g., product suggestions or purchasing procedures).

[1299] Through each of the above steps, data collected from different platforms is efficiently integrated, and personalized information is provided to users based on the results of analysis using machine learning models, thereby achieving advanced customer service and improved service.

[1300] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1301] This invention relates to a data integration management system that collects, organizes, and integrates data through APIs from different platforms. Furthermore, it analyzes the data using machine learning models and provides personalized information according to the user's needs and emotions. By combining it with an emotion engine that recognizes the user's emotions, even more advanced personalization becomes possible.

[1302] System Configuration

[1303] The system mainly consists of the following components:

[1304] 1. Data collection server: Acquires data from different platforms via API.

[1305] 2. Data integration server: Organizes and integrates collected data and eliminates duplicate data.

[1306] 3. Emotion engine: Recognizes user emotions and reflects them in data analysis.

[1307] 4. Analysis Server: Based on the integrated data, the data is analyzed using machine learning models to generate personalized information based on the user's needs and emotions.

[1308] 5. User Interface: The interface through which the user accesses personalized information.

[1309] Program processing

[1310] Data collection

[1311] The server connects to the APIs of the various platforms the user uses and collects data. For example, if the user uses cloud storage, a calendar app, an email service, or a social networking service, the server collects data from each API.

[1312] Data organization and integration

[1313] The server organizes and integrates the data collected from each platform. In doing so, it automatically eliminates duplicate data and unnecessary information. For example, it aligns posts obtained from social media with communications from email services and provides them as an integrated data set.

[1314] emotion recognition

[1315] The emotion engine recognizes the user's emotions. It analyzes user input, voice data, facial expression data, etc. to identify the emotion the user is currently feeling. This emotion data is reflected in subsequent data analysis and personalization.

[1316] Data Analysis and Personalization

[1317] The server then analyzes the integrated data using machine learning models to extract the most relevant information based on the user's past behavior, current situation, and perceived emotions, generating a personalized data set. For example, if the user is feeling stressed, it will prioritize relaxation information and entertainment content.

[1318] Providing data to users

[1319] The device provides the analyzed personalized data to the user, helping the user easily access the information they need through a user interface. For example, notifications are displayed through smartphone applications, allowing users to check important schedules, messages, and emotional content.

[1320] Specific examples

[1321] 1. The user launches the app on their smartphone and taps the "Integrate data" button.

[1322] 2. The device sends a request to the server to collect data from different platforms.

[1323] 3. The server collects data from cloud storage, calendars, email services, and social media.

[1324] 4. The server consolidates and organizes the collected data to create a consistent data set.

[1325] 5. The emotion engine recognizes the user's emotions and provides the data to the server.

[1326] 6. The server analyzes the data using machine learning models to generate a personalized dataset based on the user's needs and perceived emotions.

[1327] 7. The device provides personalized information to the user through a user interface.

[1328] 8. Users can view schedules, important emails, and emotional suggestions (such as entertainment content or relaxation methods) generated on their smartphone screen, and take action to improve productivity.

[1329] As described above, the system of the present invention efficiently integrates data from a variety of devices and cloud services, providing an environment in which users can quickly access the information they need. In particular, by combining it with an emotion engine, advanced personalization based on the user's emotional state can be achieved, making it possible to better meet individual needs. This system is expected to significantly reduce the data management burden on users, particularly business people, and improve the quality of their daily lives.

[1330] The processing flow will be explained below.

[1331] Step 1:

[1332] A user opens the application and wishes to use the data integration management system. The user launches the app on their smartphone and taps the "Integrate data" button.

[1333] Step 2:

[1334] The device sends a request to the server, conveying the user's request. The device sends an HTTP request over the internet to an API endpoint, which includes the user's credentials and a list of services they want to integrate with.

[1335] Step 3:

[1336] The server connects to the APIs of various services (e.g., cloud storage, calendar apps, email services, SNS) to collect data. The server sends an HTTP request to the service's API endpoint and temporarily stores the collected data in memory.

[1337] Step 4:

[1338] The server organizes and integrates the data it collects. For example, it combines file data collected from cloud storage and schedule data collected from a calendar app into a single integrated data set. It also eliminates duplicate and unnecessary data.

[1339] Step 5:

[1340] The emotion engine recognizes the user's emotions. The user's voice data, text data, or image data obtained from the terminal or external device is sent to the server, which inputs it into the emotion engine. The emotion engine analyzes this data and recognizes the user's emotional state (e.g., joy, sadness, stress).

[1341] Step 6:

[1342] The server uses machine learning models to analyze the integrated data. The server analyzes the data based on the user's past behavioral history, current situation, and recognized emotions, and extracts the information most appropriate for the user. For example, if the user is feeling stressed, it will prioritize information related to relaxation.

[1343] Step 7:

[1344] The server sends the parsed personalized data to the device. The server formats the data in the specified format (e.g., JSON) and sends it to the device. The data includes important schedules, emails, SNS notifications, etc.

[1345] Step 8:

[1346] The device displays the received data through a user interface, and the device displays personalized information (e.g., the latest schedule, important email notifications, and emotional entertainment content) on the application screen.

[1347] Step 9:

[1348] The user can review the provided data and take necessary actions. For example, the user can check the displayed schedule details and prepare for a meeting. The user can also reduce stress by viewing relaxing content tailored to the user's emotions.

[1349] As described above, this system collects and integrates data from the user's various devices and cloud services, and by using an emotion engine, achieves advanced personalization according to the user's emotional state, allowing users to quickly access the information they need most and supporting efficient daily life and work performance.

[1350] Example 2

[1351] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1352] Currently, collecting, integrating, and analyzing data from different sources is a time-consuming and labor-intensive task. It is also difficult to provide personalized information that takes into account the user's emotions, making it difficult to provide appropriate services that meet the user's needs. Furthermore, issues such as data duplication and organization are not taken into consideration, resulting in a lack of user convenience.

[1353] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data through application programming interfaces of different information sources, means for organizing and integrating the collected data, means for eliminating duplicate data, means for using an emotion engine to recognize the user's emotions, means for analyzing the data using a machine learning algorithm and personalizing it according to the user's needs and emotions, and means for providing the personalized data to the user. This makes it possible to efficiently collect and integrate data from multiple information sources and provide personalized information that takes the user's emotions into consideration.

[1354] "Sources" refer to the systems and platforms that provide data, including cloud storage, calendar apps, email services, and social networking services.

[1355] An "application programming interface" is a defined set of communication rules that allow software applications to interact with other applications and services.

[1356] "Data collection" refers to the process of obtaining data from different sources and storing it on a server.

[1357] Data consolidation is the process of organizing collected data into a consistent format, linking it where necessary, and eliminating duplicate data.

[1358] "Duplicate data elimination" is the process of detecting and removing duplicate data to prevent the same information from being collected multiple times.

[1359] An "emotion engine" is an algorithm or software that analyzes a user's facial expressions, text, voice, etc. to recognize emotions.

[1360] A "machine learning algorithm" is a mathematical model or method for learning patterns from large amounts of data and making predictions or classifications.

[1361] "Personalization" is the process of providing information optimized to the needs and emotions of individual users based on collected data and analysis results.

[1362] "Means for providing to the user" refers to an interface or notification system for providing the analyzed information in a form that the user can use.

[1363] This invention is a system that collects data through application programming interfaces (APIs) from different information sources, organizes, integrates, and analyzes them, and provides personalized information according to the user's needs and emotions. The main components of the system include the following:

[1364] 1. Data collection server: The server connects to the APIs of cloud storage, calendar apps, email services, social networking sites, etc. used by users to collect data. Specifically, it uses APIs such as Google Drive, Google Calendar, Gmail, and Twitter.

[1365] 2. Data integration server: The server integrates the collected data and organizes it into a consistent format. This process also eliminates duplicate data. Specifically, the server processes and formats the data using the Python libraries Pandas and NumPy.

[1366] 3. Emotion Engine: The emotion engine is designed to recognize the user's emotions. The server uses an emotion recognition API (e.g., Emotion API) to analyze the text data, voice data, and facial expression data entered by the user.

[1367] 4. Analytics Server: The server analyzes the data using machine learning algorithms to generate personalized information based on the user's past behavior, current situation, and perceived emotions. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch.

[1368] 5. User Interface: The device is responsible for providing personalized information to the user. It displays the information as notifications and messages through smartphone applications and web interfaces, making it easily accessible to the user.

[1369] For example, when a user launches a dedicated smartphone app and taps the "Integrate Data" button, the device sends a data collection request to the server. The server accesses various APIs to collect, organize, and integrate the necessary data. The emotion engine then analyzes the text and facial expression data to identify the user's emotions. The data is analyzed using machine learning algorithms to generate personalized information, which the device then provides to the user as notifications or messages. This series of processes allows users to quickly access the information they need and reduces the burden of information management.

[1370] An example prompt is:

[1371] "I want to develop a system that integrates and organizes data from different platforms, recognizes user emotions, and provides analytical data. I will use EmotionAPI for emotion recognition and TensorFlow for machine learning. Please explain the program process for this system step by step."

[1372] This system is expected to improve the quality of daily life and business by enabling users to integrate data from a variety of sources and obtain the most appropriate information based on their emotions.

[1373] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1374] Step 1:

[1375] Data collection

[1376] The server connects to the application programming interfaces (APIs) of various information sources used by the user and collects data. It receives authentication from each information source using an API key or OAuth token as input. For example, it retrieves the user's schedule data from the Google Calendar API and email data from the Gmail API. As output, it stores the collected data in JSON format on the server.

[1377] Specifically, the server sends the following API request:

[1378] GET https: / / www.googleapis.com / calendar / v3 / calendars / primary / events?key=API_KEY

[1379] This request returns the event data from Google Calendar.

[1380] Step 2:

[1381] Data organization and integration

[1382] The server organizes and consolidates the collected data. It uses data from each source stored in JSON format as input. It standardizes the data format, deletes unnecessary data, and eliminates duplicate data. Specifically, it uses Python's Pandas and NumPy libraries to handle the data as a data frame, extracts and formats the necessary information, and generates a consistent dataset as output.

[1383] Specifically, the server performs the following operations:

[1384] import pandas as pd

[1385] calendar_data = pd.read_json('calendar_data.json')

[1386] email_data = pd.read_json('email_data.json')

[1387] merged_data = pd.merge(calendar_data, email_data, on='event_id', how='outer')

[1388] This will merge your calendar and email data.

[1389] Step 3:

[1390] emotion recognition

[1391] The emotion engine recognizes the user's emotions. As input, it uses the user's text data, voice data, and facial expression data. It sends the data to an emotion recognition API to perform emotion analysis. For example, it uses Microsoft's Emotion API to obtain the emotional state from the text data. As output, an emotion score is returned to the server.

[1392] Specifically, the server sends the following API request:

[1393] POST https: / / westus.api.cognitive.microsoft.com / text / analytics / v3.0 / sentiment

[1394] This request returns a sentiment score for the text data.

[1395] Step 4:

[1396] Data Analysis and Personalization

[1397] The server uses machine learning algorithms to analyze the data and generate personalized information. It uses the cleaned and consolidated data and sentiment scores as input. It applies predictive models based on historical data and sentiment data using TensorFlow or PyTorch. As output, it generates personalized information tailored to the user's needs.

[1398] Specifically, the server performs the following operations:

[1399] import tensorflow as tf

[1400] model = tf.keras.models.load_model('model.h5')

[1401] predictions = model.predict(merged_data)

[1402] This operation generates personalized information.

[1403] Step 5:

[1404] Providing data to users

[1405] The device provides personalized information through a user interface. It uses the parsed personalized information as input, provides it to the user as notifications or messages through a smartphone application or web interface, and displays the information the user needs as output.

[1406] Specifically, the device performs the following operations:

[1407] NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);

[1408] Notification notification = new Notification.Builder(this)

[1409] .setContentTitle("Relaxation Guide")

[1410] .setContentText("Click here for a 10-minute guided meditation to help you relax right now")

[1411] .setSmallIcon(R.drawable.ic_notification)

[1412] .build();

[1413] notificationManager.notify(notificationId, notification);

[1414] This action will cause a notification to appear on the user's device.

[1415] Through the above steps, a system is realized that allows users to integrate data from a variety of information sources and provide optimal information according to their emotions.

[1416] (Application example 2)

[1417] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1418] In industrial environments, it is necessary to collect and integrate data from multiple different platforms and systems to obtain accurate information. However, there is a lack of mechanisms for efficiently collecting and integrating individual data, resulting in issues such as data duplication and the cumbersome task of organizing it. Furthermore, there is a lack of personalized support based on the emotions of workers, which prevents sufficient improvements in production efficiency and safety.

[1419] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting data through APIs of different platforms; means for organizing and integrating the collected data; means for analyzing the data using a machine learning model and providing personalized information according to the user's needs and emotions; means for generating suggestions for improving efficiency and safety in the work environment based on the integrated data; means for providing the suggestions to the user through a user interface and providing feedback in real time; and means for recognizing the user's emotions and reflecting the emotional information in the analyzed data. This makes it possible to efficiently collect data from multiple data sources and support real-time improvements in the efficiency and safety of industrial environments based on the integrated information.

[1420] A "platform API" is an interface for exchanging and manipulating data between different systems and devices.

[1421] "Means of collecting data" refers to a mechanism for obtaining the necessary information through APIs of different platforms.

[1422] "Means for organizing and integrating data" refers to a mechanism for systematically organizing collected information, eliminating duplicate data, and consolidating it into a single data set.

[1423] A "machine learning model" is an algorithm for analyzing large amounts of data and finding patterns, and is a model for making adaptive predictions and suggestions based on the results obtained from the analysis.

[1424] "Personalized information" is information that is customized according to the individual needs and feelings of the user.

[1425] The "means for generating proposals" is a mechanism for creating specific actions and proposals for users based on the analyzed data.

[1426] A "user interface" is a screen or operating means that allows a user to exchange information with a system.

[1427] "Means of providing feedback in real time" refers to a mechanism that instantly notifies the user of analysis results and suggestions, and responds immediately to the user's actions and choices.

[1428] A "means for recognizing emotions" is a mechanism for analyzing emotions from the user's facial expressions, voice, etc., and converting those emotions into data.

[1429] The present invention relates to a data integration management system for improving efficiency and safety in an industrial environment. The system is implemented using the following hardware and software.

[1430] System Configuration

[1431] The system includes the following components:

[1432] 1. Data collection server: Connects to APIs of different platforms to collect data. Specific API examples include factory sensor device APIs and surveillance camera APIs.

[1433] 2. Data integration server: Organizes the collected data, eliminates duplicates, and integrates them. A database such as MongoDB is used to generate an integrated data set.

[1434] 3. Emotion recognition engine: Recognizes emotions from the user's facial expressions and voice, and generates emotional data. Specifically, it uses the Emotion API.

[1435] 4. Analysis Server: Analyzes the integrated data using machine learning models (e.g., built using TensorFlow or PyTorch) and generates personalized information according to the user's needs and emotions.

[1436] 5. User Interface: Provide feedback to the robot using the Robot Operating System (ROS).

[1437] Processing Description

[1438] The data collection server collects data via API from factory sensor devices, surveillance cameras, inventory systems, etc. This allows the real-time data obtained from each platform to be integrated.

[1439] The data integration server organizes the collected data, eliminating duplicates and generating a consistent dataset, which is stored in MongoDB for efficient data manipulation.

[1440] The emotion recognition engine analyzes the user's facial expressions and voice data acquired from a camera or microphone to generate emotion data. This emotion data is added to the integrated data and reflected in the analysis.

[1441] The analytics server runs machine learning models using the integrated data and sentiment data to generate recommendations to improve production efficiency and safety, including real-time work efficiency improvements, break recommendations, and inventory reordering.

[1442] The user interface uses ROS to provide specific feedback, such as prompting workers to take a break or showing instructions on how to avoid hazards.

[1443] Specific examples

[1444] 1. While a user is working in a factory, a camera captures the user's facial expression data.

[1445] 2. The data collection server collects sensor data and surveillance camera video data via API.

[1446] 3. The data integration server organizes this data and generates a consistent dataset.

[1447] 4. The emotion recognition engine recognizes the user's emotions and adds them to the integrated data.

[1448] 5. The analytics server analyzes the data using machine learning models to generate personalized recommendations.

[1449] 6. The user interface provides real-time feedback to the user through ROS.

[1450] Prompt Sentence Examples

[1451] Integrate factory sensor data, surveillance camera footage, inventory data, and emotion data obtained from workers' facial expressions, analyze the data using machine learning models, and generate suggestions to improve production efficiency and work safety in the factory. Specifically, provide feedback including actions such as suggesting a break if a worker is tired, or automating reordering if inventory is low.

[1452] This system can efficiently collect data from multiple sources and use the analysis results to help improve the efficiency and safety of industrial environments in real time, which is expected to improve factory productivity and worker satisfaction.

[1453] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1454] Step 1:

[1455] Data collection

[1456] The server collects data from APIs such as factory sensor devices, surveillance cameras, and inventory systems. Although the data format obtained from each device is different, it accesses each API endpoint to obtain sensor information, video data, and inventory information. The input to this step is the raw data obtained from the API, and the output is a temporarily stored dataset. Specifically, it sends an HTTP request to the API endpoint and obtains the response data.

[1457] Step 2:

[1458] Data Integration

[1459] The server integrates the collected data in different formats and eliminates duplicates. To do this, it organizes the data using a database such as MongoDB. The input to this step is the collected raw data, and the output is an integrated dataset. Specifically, it reads the data stored in the database, matches the data structure, and performs a process to remove duplicate items.

[1460] Step 3:

[1461] emotion recognition

[1462] The server inputs image data acquired from the surveillance camera into an emotion recognition engine to analyze the user's emotions. This engine uses the Emotion API to analyze emotions from the image data and output it as data. The input for this step is image data, and the output is analyzed emotion data. Specifically, the image data is sent to the Emotion API and emotion-tagged data is received.

[1463] Step 4:

[1464] Data analysis

[1465] The server analyzes the integrated data and sentiment data using machine learning models such as TensorFlow and PyTorch. This analysis generates personalized suggestions based on the user's needs and sentiment. The inputs to this step are the integrated dataset and sentiment data, and the output is personalized suggestions. Specifically, the data is preprocessed, input to the machine learning model, and the analysis results are obtained.

[1466] Step 5:

[1467] Proposal Generation

[1468] The server generates proposals to improve the factory's production efficiency and safety based on the results of the data analysis. The input to this step is the analysis results, and the output is specific proposals. Specific operations include formatting the proposals based on the analysis results and generating actions to improve work efficiency, propose breaks, and reorder inventory.

[1469] Step 6:

[1470] Providing feedback

[1471] The server provides real-time feedback to the user through a user interface. Specifically, it uses the Robot Operating System (ROS) to notify the worker and prompt them to take specific actions. The input of this step is the generated suggestion, and the output is the feedback displayed to the user. The specific action is to communicate the suggestion to the user through ROS.

[1472] Step 7:

[1473] Monitoring User Actions

[1474] The device monitors how the user responds to the feedback. This accumulates data that can be used for future analysis. The input for this step is the user's behavioral data, and the output is saved as reaction data. Specifically, the device collects the user's behavioral log and saves it in a database that will be used for the next data analysis.

[1475] As a result, it is possible to efficiently collect information from multiple data sources and, based on the analysis results, to help improve the efficiency and safety of industrial environments in real time.

[1476] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1478] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1479] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1480] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1481] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1482] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1483] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1484] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1485] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1486] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1487] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1488] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1489] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1490] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1491] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1492] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1493] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1494] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1495] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1496] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1497] The following is further disclosed regarding the above embodiment.

[1498] (Claim 1)

[1499] A means of collecting data through APIs of different platforms,

[1500] A means of organizing and consolidating the collected data,

[1501] A means to analyze data using machine learning models and personalize it according to user needs;

[1502] A system including means for providing the personalized data to a user.

[1503] (Claim 2)

[1504] including means for merging the collected data and eliminating duplicates to generate a unified data structure;

[1505] 10. The system of claim 1.

[1506] (Claim 3)

[1507] A means for extracting and presenting high-priority information based on a user request;

[1508] 10. The system of claim 1.

[1509] "Example 1"

[1510] (Claim 1)

[1511] A means of collecting data through APIs of different platforms,

[1512] A means of organizing and consolidating the collected data,

[1513] A means to analyze data using machine learning models and personalize it according to user needs;

[1514] means for providing the personalized data to a user;

[1515] A means for the terminal to send a data integration request to the server based on a user's operation;

[1516] A means of eliminating unnecessary and redundant information from the integrated data;

[1517] means for transmitting the generated personalized data to the terminal and displaying it on a user interface;

[1518] A system including:

[1519] (Claim 2)

[1520] This includes the ability to prioritize and present important information using integrated data.

[1521] 10. The system of claim 1.

[1522] (Claim 3)

[1523] This includes the ability to use generative AI models to extract and provide information relevant to prompts based on user requests.

[1524] 10. The system of claim 1.

[1525] "Application Example 1"

[1526] (Claim 1)

[1527] A means of collecting data through APIs of different platforms,

[1528] A means of organizing and consolidating the collected data,

[1529] A means to analyze data using machine learning models and personalize it according to user needs;

[1530] means for providing the personalized data via a display device;

[1531] A system including a means for presenting personalized information to a user utilizing a visual display.

[1532] (Claim 2)

[1533] 10. The system of claim 1, further comprising means for merging the collected data and eliminating duplicates to generate a unified data structure.

[1534] (Claim 3)

[1535] 10. The system of claim 1, further comprising means for extracting high priority information based on a user request and presenting the information to the user via a display device.

[1536] "Example 2: Combining Emotion Engines"

[1537] (Claim 1)

[1538] means for collecting data through application programming interfaces of different sources;

[1539] A means of organizing and consolidating the collected data,

[1540] A means of eliminating duplicate data;

[1541] means for using an emotion engine to recognize an emotion of a user;

[1542] A means for analyzing data using machine learning algorithms and personalizing it according to the user's needs and emotions;

[1543] A system including means for providing the personalized data to a user.

[1544] (Claim 2)

[1545] including means for merging the collected data and eliminating duplicates to generate a unified data structure;

[1546] 10. The system of claim 1.

[1547] (Claim 3)

[1548] A means for extracting and presenting high-priority information based on user requests and emotion recognition;

[1549] 10. The system of claim 1.

[1550] "Application example 2 when combining emotion engines"

[1551] (Claim 1)

[1552] A means of collecting data through APIs of different platforms,

[1553] A means of organizing and consolidating the collected data,

[1554] A means for analyzing data using a machine learning model to provide personalized information according to the user's needs and emotions;

[1555] means for generating recommendations based on the integrated data to improve efficiency and safety within the work environment;

[1556] means for providing said suggestions to a user through a user interface for real-time feedback;

[1557] A means for recognizing a user's emotions and reflecting the emotional information in the analysis data;

[1558] A system including:

[1559] (Claim 2)

[1560] including means for merging the collected data and eliminating duplicates to generate a unified data structure;

[1561] 10. The system of claim 1.

[1562] (Claim 3)

[1563] This includes a means for integrating various sensor data, surveillance camera video data, inventory data, and worker emotion data in industrial environments, and extracting and presenting high-priority information.

[1564] 10. The system of claim 1. [Explanation of symbols]

[1565] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting data through APIs of different platforms, A means of organizing and integrating the collected data; A means to analyze data using machine learning models and personalize it according to user needs; A system including means for providing the personalized data to a user.

2. including means for merging the collected data and eliminating duplicates to generate a unified data structure; The system of claim 1 .

3. A means for extracting and presenting high-priority information based on a user request; The system of claim 1 .

Images