System

Abstract

A system is provided.SOLUTION: The system includes a means for capturing big data and complementing missing data, a means for finding a structure of a causal relation from the captured data by using a learning model, a means for constructing a Bayesian network based on the learned model and learning a probability relation, and a means for analyzing a factor affecting a specific event.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While conventional data analysis techniques made it possible to analyze correlations from large amounts of data, identifying causal relationships was extremely difficult. This meant that companies were unable to fully utilize data, and small and medium-sized enterprises in particular faced many challenges in equipment maintenance and troubleshooting. As a result, they were unable to quickly identify the causes of equipment anomalies and problems and take effective measures, resulting in a decline in business efficiency. [Means for solving the problem]

[0005] The present invention provides a means for importing big data and filling in missing data, a means for using a learning model to discover the structure of causal relationships from the imported data, a means for building a Bayesian network based on the learned model and learning probabilistic relationships, and a means for analyzing factors that influence specific events. The invention also ensures data integrity by acquiring big data from IoT devices, importing it, and filling in missing data. Furthermore, by utilizing a generative AI model to discover the structure of causal relationships, it is possible to identify causal relationships that were difficult to identify using conventional methods, thereby improving business efficiency and preventing problems based on insights gained from the data. This allows companies to effectively utilize data and use it to formulate business strategies and improve their operations.

[0006] "Big data" refers to a collection of large amounts of data that are difficult to handle using conventional data processing techniques, and is generally characterized by the volume, velocity, and variety of the data.

[0007] "Missing data imputation" refers to the process of filling in missing values ​​in a dataset in some way to ensure data consistency and completeness.

[0008] A "causal relationship" is a relationship in which one event or state influences another, and refers to a relationship in which the direction of the causal arrow is clearly defined.

[0009] "Discovering structure using machine learning models" means applying machine learning algorithms to understand the features and patterns in data and automatically discover hidden relationships between data.

[0010] A "Bayesian network" is a graph model based on probability theory, which represents nodes as random variables and edges as conditional dependencies, and is a method for modeling causal relationships.

[0011] A "probability relationship" refers to the relationship between the probability of one event occurring and the probability of another event occurring conditioned on that event.

[0012] An "event" refers to a specific occurrence or change of state observed within a system, and is a phenomenon of interest to users.

[0013] An "IoT device" refers to a physical device that is connected to the Internet and is capable of collecting and transmitting data.

[0014] A "generative AI model" is an artificial intelligence (AI) model that learns patterns from data and makes predictions and identifies causal relationships for new data.

[0015] "Insights" refers to the insights and knowledge gained through data analysis, which are used to improve business operations and support decision-making. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention provides a system for effectively utilizing big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[0038] 1. Data Acquisition and Preprocessing

[0039] The server captures IoT device data

[0040] The server ingests big data obtained from various IoT devices. This data usually comes in various formats and structures, so the server converts it into a unified format (e.g., CSV file) and ingests it into a DataFrame that can be analyzed.

[0041] The server completes the missing data

[0042] Since the ingested data is likely to contain missing values, the server performs preprocessing and imputes the missing values ​​in an appropriate way, for example, by using forward fill to maintain data consistency.

[0043] 2. Discovering causal structures

[0044] The server discovers the causal structure

[0045] After data preprocessing is complete, the server uses a generative AI model to discover the causal structure within the data. At this stage, advanced machine learning algorithms and statistical methods are used to uncover hidden causal relationships between data, and the resulting structural model is then used to build a Bayesian network.

[0046] 3. Bayesian Network Training

[0047] The server builds a Bayesian network and learns probability relationships.

[0048] The server builds a Bayesian network based on a structural model of causal relationships. The Bayesian network models the conditional probabilities between variables and establishes which variables influence which others. Through this learning process, the probabilistic relationships between each node (variable) are defined.

[0049] 4. Causal Analysis

[0050] The user specifies a specific event

[0051] When a user wants to analyze the causal relationship of a specific anomaly or event (e.g., temperature abnormality) in a piece of equipment, the user sends a query from the terminal to the server. This query includes details of the specific event they want to investigate.

[0052] The server analyzes the causal relationships

[0053] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. Specifically, it calculates the influence of other related nodes (variables) to identify the cause of the anomaly.

[0054] 5. Return of results and decision making

[0055] The server returns the analysis results to the device

[0056] Once the causal relationship analysis is complete, the server sends the results back to the device, allowing the user to create an appropriate maintenance plan or take appropriate measures to deal with problems.

[0057] Specific examples

[0058] For example, if a "temperature abnormality" occurs in a facility in a factory, the user uses the present invention to identify the cause. In this case, the following procedure is followed.

[0059] 1. The server collects IoT device data: The server collects sensor data (temperature, pressure, current, etc.) obtained from each piece of factory equipment and converts it into an analyzable format.

[0060] 2. The server completes missing data values: The server completes missing parts of the imported data to maintain data consistency.

[0061] 3. The server discovers the causal structure: The server uses a generative AI model to learn the causal relationships between the data and identify potential factors related to temperature anomalies.

[0062] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns a conditional probability table.

[0063] 5. The user specifies a specific event: The user sends an event called "temperature abnormality" from the terminal to the server as a query.

[0064] 6. Server analyzes causality: The server receives the query and analyzes the cause of the temperature anomaly using a Bayesian network.

[0065] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can then use the results to create an appropriate maintenance plan.

[0066] In this way, the present invention utilizes data held by a company to achieve efficient equipment management and troubleshooting.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The server imports IoT device data. Specifically, the server receives data obtained from various sensors and devices in a unified format such as CSV and converts it into an analyzable DataFrame. This data import is performed periodically, and data updates can also be made in real time.

[0070] Step 2:

[0071] The server completes missing data values. Because the imported data often contains missing values, the server performs preprocessing and completes the missing data in an appropriate manner. Specifically, it maintains data consistency by using techniques such as forward filling using previous data values.

[0072] Step 3:

[0073] The server discovers the causal structure. After preprocessing is complete, the server uses a generative AI model to explore causal relationships between the data. This involves using machine learning algorithms and statistical methods to generate a structural model that includes hidden causal relationships.

[0074] Step 4:

[0075] The server builds a Bayesian network and learns the probabilistic relationships. Based on the structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. In this learning process, the probabilistic relationships between each node are defined in detail.

[0076] Step 5:

[0077] The user specifies a specific event. The user enters a query about a specific event of interest (e.g., temperature anomaly) from the terminal and sends it to the server. This query contains details of the event they want to investigate.

[0078] Step 6:

[0079] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies other factors that may affect the event and evaluates the degree of influence.

[0080] Step 7:

[0081] The server returns the analysis results to the terminal. Once the causal relationship analysis results are obtained, the server sends the results back to the terminal. Based on these results, the user can create equipment maintenance plans and troubleshoot problems.

[0082] The above are the specific processing steps of the system of the present invention, which enable companies to effectively utilize big data and optimize their asset management through advanced causal analysis.

[0083] Example 1

[0084] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0085] Conventional systems make it difficult to efficiently utilize a company's big data and effectively carry out equipment maintenance and troubleshooting. In particular, converting data into a unified format, filling in missing data, and automatically discovering and analyzing causal relationships are often done manually, which is time-consuming and costly. It is also difficult to visualize the structure of causal relationships and quickly identify the factors that affect specific events. There is a need for a system that can solve this issue and improve the efficiency of equipment management and troubleshooting.

[0086] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0087] In this invention, the server includes means for importing big data and supplementing missing data, means for converting the imported data into a unified format and processing it into an analyzable structure, means for discovering a causal structure from preprocessed data using a generative AI model, means for building a Bayesian network based on the learned causal structure and learning probability relationships, and means for analyzing factors that influence specific events. This enables unified handling of big data in various formats held by a company, efficient data analysis, and rapid decision-making for equipment maintenance and troubleshooting.

[0088] "Big data" refers to large amounts of diverse, rapidly generated data sets that are difficult to process using traditional database management tools.

[0089] "Missing data" refers to values ​​that are missing from a dataset for some reason, and is often expressed in the form of NaN (Not a Number) or similar.

[0090] "Uniform format" refers to converting data from various formats into a consistent format (e.g., CSV, DataFrame, etc.), which makes data analysis easier.

[0091] "Analyzable structure" means that the data has a format and structure suitable for analysis, particularly a data frame or table format.

[0092] "Generative AI model" refers to a learning model generated using artificial intelligence, specifically including models built using machine learning or deep learning algorithms.

[0093] A "causal structure" refers to a model that shows the cause-and-effect relationship between data, visualizing how each element affects other elements.

[0094] A "Bayesian network" is a type of probabilistic graphical model that refers to a network used to represent conditional dependencies between variables.

[0095] A "probability relationship" refers to the conditional probability relationship between variables, and is a mathematical expression of the degree to which each variable affects other variables.

[0096] A "specific event" refers to a specific abnormality or phenomenon of interest to the user (e.g., temperature abnormality, machine failure, etc.), and causal analysis is performed based on this.

[0097] "Influencing factors" refer to elements or variables that have an effect on a particular event or outcome, and include multiple factors that may be causative.

[0098] "Analytical means" refers to the methods and processes used to perform data analysis, particularly using statistical methods and machine learning algorithms.

[0099] The present invention relates to a system that effectively utilizes big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[0100] Data acquisition and preprocessing

[0101] The server captures IoT device data

[0102] The server collects data in real time from various IoT devices. This data includes sensor data such as temperature, pressure, and current. Specifically, the server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. This operation allows the data to be managed in a consistent format, making subsequent analysis easier.

[0103] The server completes missing data

[0104] Since the imported data often contains missing values, the server uses the fillna method from the Pandas library to fill in the missing values. Specifically, it applies "dataframe.fillna(method='ffill')" to the DataFrame to fill in the missing values ​​with the previous value.

[0105] Discovery of causal structures

[0106] The server discovers the causal structure

[0107] After preprocessing is complete, the server uses a generative AI model to discover causal structures. This process utilizes machine learning libraries such as Scikit-learn and TENSORFLOW (registered trademark) to learn causal relationships based on the features of the data. Specifically, it uses a Linear Regression model and Granger Causality Tests to identify causal relationships between data.

[0108] Bayesian Network Training

[0109] The server builds a Bayesian network and learns probability relationships.

[0110] Based on the discovered causal structure, the server uses libraries such as PyMC3 and pgmpy to build a Bayesian network, which derives the conditional probability between each variable (node). Specifically, use "import pymc3 as pm" to define the model and calculate the conditional probability.

[0111] Causal analysis

[0112] The user specifies a specific event

[0113] A user sends a query about a specific anomaly or event (e.g., temperature anomaly) from their device to the server. This query includes details of the event they want to investigate. For example, a user might enter "Please analyze the cause of the temperature anomaly from the sensor data from Factory A" through a web or mobile application and submit it.

[0114] The server analyzes the causal relationships

[0115] Based on the query received from the user, the server analyzes the cause of the event using the pre-built Bayesian network. It calculates the influence from other related nodes and identifies the cause of the anomaly. Specifically, it applies "pm.sample_posterior_predictive(trace, model=model)" to calculate the probability distribution of the anomaly factors.

[0116] Returning results and making decisions

[0117] The server returns the analysis results to the device

[0118] Once the analysis is complete, the server returns the results to the user's device as a JSON-formatted HTTP response. Based on these analysis results, the user can consider appropriate maintenance plans and troubleshooting measures.

[0119] This invention enables the unified handling of big data in various formats held by companies and efficient data analysis, thereby enabling rapid and accurate decision-making regarding equipment maintenance and troubleshooting.

[0120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0121] Step 1:

[0122] Data Ingestion

[0123] The server collects data in real time from various IoT devices. As input, it uses information from each IoT device's API endpoint and message queue (e.g., MQTT). The server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. The output is a CSV file in a unified format.

[0124] Specific behavior:

[0125] Get data from the IoT device using requests.get(api_endpoint).

[0126] Convert to CSV using data.to_csv('path / to / file.csv') and save.

[0127] Step 2:

[0128] Data Preprocessing

[0129] The server imputes missing values ​​in the imported data. The input is the CSV file obtained in step 1. The server uses the fillna method of the Pandas library to fill missing values ​​with the previous value. The output is a data frame with imputed missing values.

[0130] Specific behavior:

[0131] Read the CSV file with dataframe = pd.read_csv('path / to / file.csv').

[0132] Fill missing values ​​with dataframe.fillna(method='ffill').

[0133] Step 3:

[0134] Discovery of causal structures

[0135] Based on the preprocessed data, the server uses a generative AI model to discover the causal structure. The input is the data frame obtained in step 2. Causal relationships are learned using libraries such as Scikit-learn and TensorFlow. The output is a learned model that shows the causal relationships.

[0136] Specific behavior:

[0137] Define the model with model = LinearRegression().

[0138] Learn causal relationships with model.fit(X, y).

[0139] Step 4:

[0140] Bayesian Network Training

[0141] The server builds a Bayesian network based on the structural model of causal relationships and learns conditional probabilities. The input is the causal relationship model created in step 3. The Bayesian network is built using the PyMC3 and pgmpy libraries. The output is a Bayesian network probabilistic model.

[0142] Specific behavior:

[0143] Define the Bayesian network with pm.Model() as model:

[0144] Train a probabilistic model with pm.sample().

[0145] Step 5:

[0146] Receiving specific event analysis queries

[0147] The user sends a query about a specific anomaly or event (e.g., temperature anomaly) from a terminal to the server in the form of a prompt sentence. The input is the query text entered by the user. The output is the query sent to the server.

[0148] Specific behavior:

[0149] The user enters the following into the form on the terminal: "Please analyze the cause of the temperature abnormality from the sensor data of Factory A."

[0150] Click the submit button to send the query to the server.

[0151] Step 6:

[0152] Causal analysis

[0153] Based on the received query, the server analyzes the cause of the specified event using a Bayesian network. The inputs are the Bayesian network probabilistic model obtained in step 4 and the query received in step 5. The output is the analysis result of the anomaly cause.

[0154] Specific behavior:

[0155] Perform inference using pm.sample_posterior_predictive(trace, model=model).

[0156] Based on the query, determine the cause of the event.

[0157] Step 7:

[0158] Return of analysis results

[0159] The server returns the analysis results to the user's device. The input is the analysis results obtained in step 6. The output is the JSON data of the analysis results sent to the user.

[0160] Specific behavior:

[0161] Convert the parsed results into JSON format.

[0162] Return the response to the user device using response = HttpResponse(json.dumps(analysis result), content_type='application / json').

[0163] summary

[0164] In this way, the system automates the entire process from data collection to the return of analysis results, enabling efficient equipment management and troubleshooting.

[0165] (Application example 1)

[0166] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] There is a need to utilize big data in factory equipment to improve the efficiency of equipment maintenance and troubleshooting. However, currently, it is difficult to process and analyze huge amounts of data, and it is difficult to analyze missing data and causal relationships. Therefore, a system that can process data more efficiently and accurately and analyze anomalies is needed.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0169] In this invention, the server includes a means for importing big data and supplementing missing data, a means for discovering a causal structure from the imported data using a learning model, a means for constructing a Bayesian network based on the learned model and learning probability relationships, a means for analyzing factors that influence specific events, a means for collecting sensor data from robots in a factory, and a means for identifying abnormalities in the robots based on the collected data and recommending a maintenance plan, thereby enabling efficient and accurate data processing and abnormality analysis.

[0170] "Big data" refers to the collection of huge amounts of data that are difficult to process and analyze using conventional methods.

[0171] "Missing data" is data that indicates a lack of some of the information available in a dataset.

[0172] A "learning model" is an algorithm that learns patterns and relationships from data and makes predictions and classifications.

[0173] "Causality" is a concept that indicates that one event has a cause-and-effect relationship with another event.

[0174] A "Bayesian network" is a model that expresses probabilistic causal relationships using a graph structure, and is a method for expressing relationships between variables based on conditional probability.

[0175] An "IoT device" is a physical device connected to the Internet, equipped with sensors and actuators and capable of collecting and transmitting data.

[0176] A "factory robot" is a machine designed to perform automated tasks in a factory environment.

[0177] "Sensor data" is data that expresses physical phenomena and environmental conditions measured by a sensor as numerical values.

[0178] A "generative AI model" is an AI model that uses machine learning algorithms to perform generative tasks (e.g., text generation or image generation).

[0179] A "maintenance plan" is a plan for regular inspections and repairs to keep facilities and equipment in good condition.

[0180] "Identifying anomalies" means detecting behavior or conditions that deviate from normal operating conditions and clarifying the cause.

[0181] The present invention provides a system for implementing efficient maintenance and troubleshooting by utilizing big data obtained from robots and equipment in a factory. Specific embodiments of the present invention will be described below.

[0182] 1. Data Collection and Preprocessing

[0183] Data collection

[0184] The server collects sensor data from IoT devices in the factory, including data on the equipment's status, such as temperature, pressure, current, and vibration, and transmits the data to the server in real time over the network.

[0185] Data Preprocessing

[0186] Since the collected data may contain missing values, the server preprocesses the data. Specifically, it uses the pandas library to convert the data into DataFrame format and imputes missing values. In this process, it uses forward fill, which fills in the previous value.

[0187] 2. Discovering causal structures

[0188] Discovering causal relationships using generative AI models

[0189] Based on the preprocessed data, the server uses a generative AI model to discover the structure of causal relationships between the data. An example of a generative AI model is the GPT model from OpenAI (registered trademark). A prompt sentence is sent to this model to analyze the causal relationships.

[0190] Example prompt sentence:

[0191] "Please find the causal relationship of the abnormal occurrence of the factory robot based on the following data:

[0192] data:

[0193] {

[0194] "timestamp": ["2023-10-01 12:00:00", "2023-10-01 12:01:00", "2023-10-01 12:02:00"],

[0195] "temperature": [45, 47, 48],

[0196] "pressure": [101, 102, 100],

[0197] "current": [5.5, 5.6, 5.3]

[0198] }

[0199] "

[0200] 3. Building a Bayesian network

[0201] Building Bayesian networks based on structure discovery

[0202] Based on the generated causal relationships, the server constructs a Bayesian network, which models the probabilistic relationships between variables and describes the impact of certain events on others.

[0203] 4. Analyze abnormalities and recommend maintenance plans

[0204] Anomaly analysis

[0205] When a user specifies a specific abnormal event (e.g., abnormal temperature) on their smartphone, the server uses a Bayesian network to analyze the cause of the abnormality and identify the main factors.

[0206] Notification of results and recommended maintenance plans

[0207] The analysis results are sent to the user's smartphone, allowing them to plan and carry out maintenance in a timely manner.

[0208] Hardware and software used

[0209] Hardware

[0210] IoT devices (sensors, actuators, etc.)

[0211] Server (high performance computer)

[0212] Smartphone device

[0213] software

[0214] pandas library (data preprocessing)

[0215] OpenAI's generative AI model (discovering causal structures)

[0216] Bayesian network construction software

[0217] By implementing this invention, it is possible to efficiently process and analyze the huge amount of data obtained from robots and equipment in a factory, identify the cause of abnormalities, and create an appropriate maintenance plan, thereby reducing the risk of equipment failure and improving operational efficiency.

[0218] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0219] Step 1: Collect data

[0220] The server collects sensor data from IoT devices in the factory (for example, robots and their peripherals). Input data includes temperature, pressure, current, vibration, etc., and this data is sent to the server in real time via the network. Specifically, the server periodically requests data from each sensor and aggregates it.

[0221] Step 2: Preprocessing the data

[0222] The server converts the received raw data into DataFrame format using the pandas library. Since the data may contain missing values, it imputes the missing values. Specifically, the server applies forward fill to the missing parts of the dataframe to create a consistent dataset. The output is clean data with missing values ​​imputed.

[0223] Step 3: Discover the causal structure

[0224] The server uses the preprocessed data to send a prompt to the generative AI model (e.g., GPT model). The prompt contains the JSON formatted content of the sensor data. The input is the clean data and the prompt, which the server sends to the generative AI model. The output is structural information about the causal relationships.

[0225] Step 4: Building a Bayesian network

[0226] The server constructs a Bayesian network based on the structural information of causal relationships obtained from the generative AI model. Specifically, the server calculates the conditional probability between each variable and expresses this in a graph structure. The input is the structural information of causal relationships, and the output is the constructed Bayesian network.

[0227] Step 5: Analyze the anomaly

[0228] The user specifies a specific abnormal event (e.g., abnormal temperature) from their smartphone device and sends a query to the server. The server then uses a Bayesian network to analyze the input abnormal event. Specifically, the server analyzes the related nodes in the Bayesian network and identifies the cause of the abnormality. The input is the query from the user, and the output is the result of identifying the cause of the abnormality.

[0229] Step 6: Communicate results and recommend a maintenance plan

[0230] The server notifies the user's smartphone of the analysis results it has obtained. The notification includes the cause of the anomaly and specific maintenance instructions accordingly. Specifically, the server formats the analysis results as a text message and sends it to the user's device. The input is the anomaly analysis result, and the output is a notification message to the user.

[0231] This will enable real-time monitoring of robots and equipment within the factory, early detection of abnormalities, and appropriate maintenance measures.

[0232] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0233] The present invention is a system that combines a system for collecting big data, completing missing data, and analyzing causal relationships with an emotion engine that recognizes user emotions. Specific embodiments of the present invention will be described below.

[0234] 1. Data Acquisition and Preprocessing

[0235] The server collects IoT device data and emotion data.

[0236] The server takes in data from various IoT devices and user emotion data obtained through the emotion engine. This data is converted into a unified format (e.g., CSV file) and saved as a DataFrame that can be analyzed.

[0237] The server completes the missing data

[0238] If there are gaps in the imported data, the server will fill them in using an appropriate method, such as forward filling using the previous data value, to maintain data consistency.

[0239] 2. Discovering causal structures

[0240] The server discovers the causal structure

[0241] The server then uses generative AI models on the preprocessed data to discover causal structures, using machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[0242] 3. Building and training a Bayesian network

[0243] The server builds a Bayesian network and learns probability relationships.

[0244] The server builds a Bayesian network based on a structural model of causal relationships and learns the conditional probabilities between each piece of data. This learning process defines in detail the probabilistic relationships between each node.

[0245] 4. Causal Analysis

[0246] The user specifies a specific event

[0247] The user enters a query about a particular event of interest (e.g., a temperature anomaly or a particular emotional state) and sends it to the server. This query includes details of the event they want to investigate.

[0248] The server analyzes the causal relationships

[0249] Based on the query received, the server uses a Bayesian network to analyze the causal relationships of the specified events, specifically by identifying other variables that may affect a particular event and assessing the degree of their influence.

[0250] 5. Return of results and decision making

[0251] The server returns the analysis results to the device

[0252] Once the results of the causal analysis are obtained, the server sends the results back to the device, allowing the user to create appropriate maintenance plans and troubleshooting measures based on these results.

[0253] Specific examples

[0254] For example, suppose a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken:

[0255] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data obtained via the emotion engine, and converts the data into an analyzable format.

[0256] 2. The server imputes missing values ​​in the data: The server imputes missing values ​​in the dataset using previous values ​​to maintain data consistency.

[0257] 3. The server discovers causal structures: The server uses a generative AI model to analyze the causal relationships in the data and identify the causal relationships related to temperature anomalies and employees' emotional states (stress levels).

[0258] 4. The server constructs a Bayesian network and learns the probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between the data.

[0259] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[0260] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[0261] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[0262] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[0263] The processing flow will be explained below.

[0264] Step 1:

[0265] The server takes in IoT device data and user emotion data. Specifically, the server receives sensor data from factories and emotion data from wearable devices worn by employees. It converts this data into a unified format (e.g., CSV file) and saves it in an analyzable DataFrame.

[0266] Step 2:

[0267] The server completes missing data. Because the imported data is likely to contain missing values, the server completes the missing values ​​in an appropriate manner. Specifically, the server maintains data consistency by using methods such as forward filling with data values ​​from previous points in time.

[0268] Step 3:

[0269] The server discovers the causal structure. Using the preprocessed data as input, the server uses a generative AI model to discover the causal structure between each piece of data. This process uses machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[0270] Step 4:

[0271] The server builds a Bayesian network and learns the probabilistic relationships. Based on a structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. This learning process defines in detail the probabilistic relationships between each node (variable).

[0272] Step 5:

[0273] The user specifies a specific event. The user inputs a specific event of interest (e.g., a query about a temperature anomaly or a specific emotional state) from their device and sends it to the server. This query contains details of the event they want to investigate.

[0274] Step 6:

[0275] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies the influence of other variables related to the event and evaluates the degree of influence.

[0276] Step 7:

[0277] The server returns the analysis results to the device. Once the results of the causal relationship analysis are obtained, the server sends the results back to the device. Based on the analysis results, the user can create appropriate maintenance plans and troubleshooting measures.

[0278] Programmatic example

[0279] For example, if a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees, the user can do so as follows.

[0280] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data (e.g., stress level, heart rate, etc.) obtained via the emotion engine, and converts the data into an analyzable format.

[0281] 2. The server completes missing data: The server completes missing values ​​in the imported data with data values ​​from the previous point in time to maintain consistency.

[0282] 3. The server discovers causal structures: The server uses a generative AI model to analyze causal relationships between data and identify causal relationships related to temperature anomalies and employee stress levels.

[0283] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between data.

[0284] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[0285] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[0286] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[0287] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[0288] Example 2

[0289] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0290] Conventional big data analysis systems not only require a huge amount of time and effort to import and preprocess data and discover causal relationships, but also insufficiently utilize emotional data. Furthermore, the process for users to analyze causal relationships based on specific events and gain actionable insights is ineffective. This makes it difficult to make appropriate decisions and efficiently manage facilities.

[0291] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for importing big data and complementing missing data, means for discovering a causal structure from the imported data using a generative AI model, means for constructing a Bayesian network based on the learned model and learning probabilistic relationships, means for a user to input a query regarding a specific event and for the server to analyze the causal relationships of the event, means for returning the analysis results to the user's terminal, and means for importing and analyzing emotion data. This enables a user to efficiently integrate big data and emotion data, analyze causal relationships based on specific events, and obtain practical insights in real time.

[0292] "Big data" refers to a set of data that is so large and complex that it cannot be processed using conventional data processing technology.

[0293] "Missing data" refers to data in a dataset where some values ​​are missing or incomplete.

[0294] A "generative AI model" refers to an artificial intelligence model that is used to learn from data and generate new predictions and insights.

[0295] A "Bayesian network" refers to a graphical model for representing causal relationships between random variables and making inferences using conditional probabilities.

[0296] "Emotional data" refers to data that represents the user's emotional state (e.g., stress, satisfaction, fatigue, etc.).

[0297] A "query" is a question or request made to a database or system to retrieve or manipulate information.

[0298] "Causality" refers to the direct effect or relationship that a change in one variable has on another variable.

[0299] "Analysis" refers to the process of examining data in detail and uncovering hidden meanings and relationships within it.

[0300] "User" refers to a person or organization that uses the system.

[0301] "Terminal" refers to a device (e.g., PC, tablet, smartphone, etc.) that a user uses to access and operate the system.

[0302] The present invention relates to a system that collects big data, integrates user emotion data, and analyzes causal relationships. This system incorporates big data, complements missing data, discovers causal relationship structures, builds Bayesian networks, analyzes causal relationships based on specific events, returns analysis results, and analyzes emotion data. Specific embodiments of the present invention are described in detail below.

[0303] Data acquisition and preprocessing

[0304] The server collects IoT device data and emotion data.

[0305] The server collects various data using the factory's sensors (e.g., temperature sensors, humidity sensors, current sensors, etc.) and emotion engines (e.g., cameras and emotion recognition software). This data is converted to, for example, CSV format and saved as a DataFrame using Python's Pandas library. Specifically, the data is read using pd.read_csv('sensor_data.csv') and converted into a unified format.

[0306] The server converts the data into a unified format and stores it.

[0307] The collected data is converted into a unified format (e.g., date and time, sensor values), for example by unifying all timestamps to UTC and scaling data in different units. This converted data is saved as a DataFrame for further processing.

[0308] The server completes the missing data

[0309] Missing values ​​in the dataset are imputed using the Pandas fillna method, e.g., dataframe.fillna(method='ffill', inplace=True) , which imputes missing values ​​with the previous value and ensures data consistency.

[0310] Discovery of causal structures

[0311] The server uses a generative AI model to discover the causal structure

[0312] Once the preprocessing is complete, a generative AI model (e.g., a causal impact model) is used to discover the causal structure of the data. Specifically, a causal analysis algorithm is run as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period) .

[0313] Building and training Bayesian networks

[0314] The server builds a Bayesian network

[0315] Use the pgmpy library to build a Bayesian network based on a structural model. Define the network as follows: model = BayesianNetwork(structure) .

[0316] The server learns the probability relationships

[0317] A Bayesian network uses conditional probabilities to learn the probabilistic relationships between nodes. Specifically, it is trained using a data frame, such as model.fit(dataframe) .

[0318] Causal analysis and return of results

[0319] A user enters a query about a specific event

[0320] A user sends a query about a specific event (e.g., temperature abnormality or a specific emotional state) to the server via a terminal. For example, a user enters a query to investigate "employee stress levels when the temperature exceeds 30 degrees."

[0321] The server analyzes the causal relationships and returns the analysis results.

[0322] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. The analysis results are sent back to the user's device. Specifically, the server uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) .

[0323] Emotional data capture and analysis

[0324] The server captures and analyzes the emotional data.

[0325] Emotion data obtained from the emotion engine is collected in real time using IP cameras and emotion recognition software, which analyzes employees' emotional states (e.g., stress, satisfaction, fatigue) from their facial expressions and voice data.

[0326] Specific examples

[0327] For example, consider a case where a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken: The server imports sensor data (e.g., temperature, humidity, current, etc.) from each piece of factory equipment and employee emotion data obtained via an emotion engine. Then, it preprocesses the data and appropriately imputes missing data. Next, it uses a generative AI model to analyze the causal relationships in the data, constructs a Bayesian network, and learns the probabilistic relationships. The user sends a query to the server from their device to investigate the events "temperature abnormalities" and "employee stress levels," and the server analyzes the causal relationships using a Bayesian network. Finally, the analysis results are sent back to the user's device, allowing appropriate maintenance measures and employee care plans to be developed.

[0328] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0329] Step 1: Ingest your data

[0330] The server collects real-time data from various factory sensors (temperature sensors, humidity sensors, current sensors, etc.) and the emotion engine (camera and emotion recognition software). The input is the raw data from each sensor and the data obtained by the emotion engine, which is converted to CSV format and saved. The data is read as a DataFrame using the Pandas library, e.g., pd.read_csv('sensor_data.csv'). The output is a unified format DataFrame.

[0331] Step 2: Preprocessing the data

[0332] The server converts the read data into a unified format. Specifically, it converts all timestamps to UTC time and unifies units as necessary. The input is various sensor data and emotion data in CSV format, and it scales the data and converts timestamps. The output is a preprocessed DataFrame.

[0333] Step 3: Imputing missing data

[0334] The server uses Pandas' fillna method to fill missing values ​​in the dataset. Specifically, dataframe.fillna(method='ffill', inplace=True) fills missing parts with the previous value. The input is a preprocessed DataFrame, and the output is a DataFrame with missing data filled in.

[0335] Step 4: Discover the causal structure

[0336] The server uses the data after preprocessing and missing data imputation to discover the causal structure using a generative AI model. Specifically, it generates a model as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period). The input is a DataFrame with missing data imputed, and the output is a model showing the causal relationships.

[0337] Step 5: Building a Bayesian network

[0338] The server uses the pgmpy library to build a Bayesian network based on a causal model. Specifically, the network is defined as model = BayesianNetwork(structure) . The input is the causal model, and the output is the structure of the Bayesian network.

[0339] Step 6: Learning Probability Relationships

[0340] The server uses a Bayesian network to learn the conditional probabilities between each node. Specifically, training is performed using a data frame using model.fit(dataframe) . The input is a DataFrame with the Bayesian network structure and missing data imputed, and the output is a Bayesian network with learned conditional probabilities.

[0341] Step 7: Enter your query

[0342] A user uses a device to send a query about a specific event (e.g., temperature anomalies or a specific emotional state) to the server. The input is the user-specified event query, and the output is the data sent to the server for analysis of the query.

[0343] Step 8: Causal analysis

[0344] The server uses a Bayesian network to analyze the causal relationships of specified events based on the query submitted by the user. For example, it uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) . The input is the user's query and a Bayesian network trained on conditional probabilities, and the output is the analysis result.

[0345] Step 9: Returning the analysis results

[0346] The server returns the analysis results to the user's device. The user can use this information to formulate appropriate maintenance measures and employee care plans. The input is the analysis results, and the output is the analysis results returned to the user's device.

[0347] Specific prompt examples:

[0348] "Analyze the fluctuations in employees' stress levels when the temperature exceeds 30 degrees."

[0349] (Application example 2)

[0350] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0351] Conventional advertising systems were unable to generate optimal ads by comprehensively analyzing users' behavioral patterns and emotional states, resulting in limited advertising effectiveness. Furthermore, there were problems with reduced targeting accuracy when acquired data was incomplete or causal analysis was insufficient. Furthermore, the lack of a mechanism for evaluating advertising effectiveness in real time and dynamically feeding back the results limited the efficiency of the system.

[0352] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0353] In this invention, the server includes means for importing big data and supplementing missing data, means for discovering a causal structure from the imported data using a learning model, means for constructing a Bayesian network based on the learned model and learning probability relationships, means for collecting and analyzing user behavioral data and emotional data, means for generating and delivering optimal advertisements based on the user's behavioral patterns and emotional state using the Bayesian network, and means for evaluating the effectiveness of the delivered advertisements and feeding back the results to optimize the advertisement content. This enables highly accurate advertisement targeting and real-time evaluation based on the user's behavioral patterns and emotional state.

[0354] "Big data" refers to large, complex, and diverse data sets, as well as the technologies and methods for efficiently processing and analyzing them.

[0355] "Missing data" refers to a portion of a dataset where necessary information is missing, or the absence of a particular data point.

[0356] A "learning model" is a mathematical model that uses machine learning algorithms to learn patterns in data and make predictions or classifications.

[0357] A causal relationship is a relationship in which one event or variable influences another.

[0358] A "Bayesian network" is a type of probabilistic graphical model used to represent conditional dependencies between variables.

[0359] "User behavioral data" refers to data about a user's behavior when using a specific device or service, including, for example, location information and app usage history.

[0360] "Emotional data" refers to data that expresses a user's emotional state as numerical values ​​or categories, and is obtained by analyzing facial expressions, tone of voice, etc.

[0361] "Advertising" refers to information promoting a particular product or service, and refers to promotions or messages sent to users.

[0362] "Targeting" refers to the technology and methods of pinpointing and delivering the most appropriate advertisements and information to specific groups or individuals.

[0363] "Real-time evaluation" refers to the process of evaluating and analyzing specific events or data the moment they occur.

[0364] "Feedback" refers to information used to modify or optimize the next action or process based on the results obtained from an action or process that has already been performed.

[0365] MODE FOR CARRYING OUT THE INVENTION

[0366] The present invention provides a system that integrates and analyzes big data and user emotional data, and generates and delivers optimal advertisements based on the user's behavioral patterns and emotional state. This system includes the following means and processing steps.

[0367] System Configuration

[0368] Hardware and software configuration

[0369] 1. Hardware:

[0370] Smartphone: Collects user behavioral data (e.g., location information, app usage history) and emotional data through the camera.

[0371] Server: Preprocesses data, analyzes causal relationships, builds Bayesian networks, and generates and delivers advertisements.

[0372] 2. Software:

[0373] Data acquisition module: The data collected from the smartphone is acquired by the server.

[0374] Preprocessing module: Imputing missing data and data transformation.

[0375] Causal analysis module: Analyzes causal relationships between data using generative AI models.

[0376] Bayesian Network Module: Constructs and learns a Bayesian network based on the discovered causal relationships.

[0377] Advertisement generation and delivery module: Generates optimal advertisements based on the user's behavioral patterns and emotional state and delivers them to the smartphone.

[0378] Effectiveness evaluation module: Evaluates the effectiveness of delivered ads in real time and optimizes the ad content based on the results.

[0379] System Processing and Data Flow

[0380] 1. Data Acquisition and Preprocessing

[0381] The server collects location information, app usage history, and emotion data obtained from the camera in real time from the smartphone, converts it into a unified format (e.g., CSV file), and stores it as an analyzable DataFrame after completing a missing data imputation process.

[0382] 2. Causal analysis

[0383] The server uses the generative AI model on the data after data ingestion to identify causal relationships between the data. Specifically, it uses machine learning algorithms and statistical methods to discover hidden causal relationships.

[0384] 3. Building and training a Bayesian network

[0385] Based on the discovered causal structure, a Bayesian network is constructed to learn the conditional probabilities between each data point. This process defines the probabilistic relationships between variables in detail.

[0386] 4. Ad generation and delivery

[0387] If the server determines that the user is in a specific behavioral pattern or emotional state, it uses a Bayesian network to generate optimal advertisements and deliver them to the smartphone. For example, if the server determines that the user is having fun, it will deliver a coupon advertisement for a nearby cafe.

[0388] 5. Evaluation and feedback of advertising effectiveness

[0389] The effectiveness of delivered ads is evaluated in real time and ad content is optimized based on collected feedback data, thereby improving the accuracy of ad targeting.

[0390] Specific examples

[0391] For example, when a user is in a shopping mall, a smartphone app can analyze the user's emotional state through the camera. If the app determines that the user is having fun, it will deliver coupon advertisements for nearby cafes. If the user is tired, it will deliver advertisements for relaxation services.

[0392] Example prompt sentence:

[0393] "What kind of coupon ads can be delivered to users at the shopping mall to get the most impact when they are having fun?"

[0394] "When a user is tired in a shopping mall, what kind of relaxation service advertisement should be delivered to them to get the most effect?"

[0395] In this way, optimal advertisement delivery based on the user's emotional state is possible.

[0396] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0397] Step 1:

[0398] The server receives location information, app usage history, and emotional data obtained from the camera in real time from the smartphone. This data is converted into a standardized format (e.g., CSV file) and stored in a database.

[0399] Input: Location information from smartphone, app usage history, emotional data

[0400] Output: Data saved in a unified format (CSV file)

[0401] Step 2:

[0402] If there are missing parts in the imported data, the server will fill in the missing data by, for example, applying a forward fill method using the previous data value or a mean value interpolation method.

[0403] Input: Imported data, including missing data

[0404] Output: A complete and consistent dataset

[0405] Step 3:

[0406] The server analyzes the enriched dataset and uses a generative AI model to discover causal structures, a process that employs machine learning algorithms and statistical methods.

[0407] Input: Imputed dataset

[0408] Output: A structural model of the discovered causal relationships

[0409] Step 4:

[0410] The server constructs a Bayesian network based on a structural model of the discovered causal relationships and learns the conditional probabilities between each piece of data.

[0411] Input: structural model of causality

[0412] Output: Constructed Bayesian network and learned conditional probabilities

[0413] Step 5:

[0414] The server analyzes specific behavioral patterns and emotional states of users and generates optimal advertisements using a Bayesian network. It selects appropriate targeted advertisements based on the latest user data obtained from smartphones.

[0415] Input: User behavior patterns, emotional state, and Bayesian network

[0416] Output: The best ad generated

[0417] Step 6:

[0418] The server then delivers the generated advertisements to smartphones, adjusting the timing to ensure they are delivered at the appropriate time based on the user's current location and behavior.

[0419] Input: Generated ad, user's current location

[0420] Output: Advertisement delivered to the user's smartphone

[0421] Step 7:

[0422] The server evaluates the effectiveness of the delivered advertisements in real time and analyzes the collected feedback data. It collects user response data (click-through rate, duration of visit, etc.) and optimizes the advertisement content based on that data.

[0423] Input: Feedback data on advertising effectiveness

[0424] Output: Optimized advertising content and advertising strategies

[0425] This enables highly accurate ad delivery based on the user's behavioral patterns and emotional state, maximizing advertising effectiveness.

[0426] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0428] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0429] [Second embodiment]

[0430] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0431] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0433] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0437] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0438] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0439] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0440] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0441] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0442] The present invention provides a system for effectively utilizing big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[0443] 1. Data Acquisition and Preprocessing

[0444] The server captures IoT device data

[0445] The server ingests big data obtained from various IoT devices. This data usually comes in various formats and structures, so the server converts it into a unified format (e.g., CSV file) and ingests it into a DataFrame that can be analyzed.

[0446] The server completes the missing data

[0447] Since the ingested data is likely to contain missing values, the server performs preprocessing and imputes the missing values ​​in an appropriate way, for example, by using forward fill to maintain data consistency.

[0448] 2. Discovering causal structures

[0449] The server discovers the causal structure

[0450] After data preprocessing is complete, the server uses a generative AI model to discover the causal structure within the data. At this stage, advanced machine learning algorithms and statistical methods are used to uncover hidden causal relationships between data, and the resulting structural model is then used to build a Bayesian network.

[0451] 3. Bayesian Network Training

[0452] The server builds a Bayesian network and learns probability relationships.

[0453] The server builds a Bayesian network based on a structural model of causal relationships. The Bayesian network models the conditional probabilities between variables and establishes which variables influence which others. Through this learning process, the probabilistic relationships between each node (variable) are defined.

[0454] 4. Causal Analysis

[0455] The user specifies a specific event

[0456] When a user wants to analyze the causal relationship of a specific anomaly or event (e.g., temperature abnormality) in a piece of equipment, the user sends a query from the terminal to the server. This query includes details of the specific event they want to investigate.

[0457] The server analyzes the causal relationships

[0458] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. Specifically, it calculates the influence of other related nodes (variables) to identify the cause of the anomaly.

[0459] 5. Return of results and decision making

[0460] The server returns the analysis results to the device

[0461] Once the causal relationship analysis is complete, the server sends the results back to the device, allowing the user to create an appropriate maintenance plan or take appropriate measures to deal with problems.

[0462] Specific examples

[0463] For example, if a "temperature abnormality" occurs in a facility in a factory, the user uses the present invention to identify the cause. In this case, the following procedure is followed.

[0464] 1. The server collects IoT device data: The server collects sensor data (temperature, pressure, current, etc.) obtained from each piece of factory equipment and converts it into an analyzable format.

[0465] 2. The server completes missing data values: The server completes missing parts of the imported data to maintain data consistency.

[0466] 3. The server discovers the causal structure: The server uses a generative AI model to learn the causal relationships between the data and identify potential factors related to temperature anomalies.

[0467] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns a conditional probability table.

[0468] 5. The user specifies a specific event: The user sends an event called "temperature abnormality" from the terminal to the server as a query.

[0469] 6. Server analyzes causality: The server receives the query and analyzes the cause of the temperature anomaly using a Bayesian network.

[0470] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can then use the results to create an appropriate maintenance plan.

[0471] In this way, the present invention utilizes data held by a company to achieve efficient equipment management and troubleshooting.

[0472] The processing flow will be explained below.

[0473] Step 1:

[0474] The server imports IoT device data. Specifically, the server receives data obtained from various sensors and devices in a unified format such as CSV and converts it into an analyzable DataFrame. This data import is performed periodically, and data updates can also be made in real time.

[0475] Step 2:

[0476] The server completes missing data values. Because the imported data often contains missing values, the server performs preprocessing and completes the missing data in an appropriate manner. Specifically, it maintains data consistency by using techniques such as forward filling using previous data values.

[0477] Step 3:

[0478] The server discovers the causal structure. After preprocessing is complete, the server uses a generative AI model to explore causal relationships between the data. This involves using machine learning algorithms and statistical methods to generate a structural model that includes hidden causal relationships.

[0479] Step 4:

[0480] The server builds a Bayesian network and learns the probabilistic relationships. Based on the structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. In this learning process, the probabilistic relationships between each node are defined in detail.

[0481] Step 5:

[0482] The user specifies a specific event. The user enters a query about a specific event of interest (e.g., temperature anomaly) from the terminal and sends it to the server. This query contains details of the event they want to investigate.

[0483] Step 6:

[0484] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies other factors that may affect the event and evaluates the degree of influence.

[0485] Step 7:

[0486] The server returns the analysis results to the terminal. Once the causal relationship analysis results are obtained, the server sends the results back to the terminal. Based on these results, the user can create equipment maintenance plans and troubleshoot problems.

[0487] The above are the specific processing steps of the system of the present invention, which enable companies to effectively utilize big data and optimize their asset management through advanced causal analysis.

[0488] Example 1

[0489] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0490] Conventional systems make it difficult to efficiently utilize a company's big data and effectively carry out equipment maintenance and troubleshooting. In particular, converting data into a unified format, filling in missing data, and automatically discovering and analyzing causal relationships are often done manually, which is time-consuming and costly. It is also difficult to visualize the structure of causal relationships and quickly identify the factors that affect specific events. There is a need for a system that can solve this issue and improve the efficiency of equipment management and troubleshooting.

[0491] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0492] In this invention, the server includes means for importing big data and supplementing missing data, means for converting the imported data into a unified format and processing it into an analyzable structure, means for discovering a causal structure from preprocessed data using a generative AI model, means for building a Bayesian network based on the learned causal structure and learning probability relationships, and means for analyzing factors that influence specific events. This enables unified handling of big data in various formats held by a company, efficient data analysis, and rapid decision-making for equipment maintenance and troubleshooting.

[0493] "Big data" refers to large amounts of diverse, rapidly generated data sets that are difficult to process using traditional database management tools.

[0494] "Missing data" refers to values ​​that are missing from a dataset for some reason, and is often expressed in the form of NaN (Not a Number) or similar.

[0495] "Uniform format" refers to converting data from various formats into a consistent format (e.g., CSV, DataFrame, etc.), which makes data analysis easier.

[0496] "Analyzable structure" means that the data has a format and structure suitable for analysis, particularly a data frame or table format.

[0497] "Generative AI model" refers to a learning model generated using artificial intelligence, specifically including models built using machine learning or deep learning algorithms.

[0498] A "causal structure" refers to a model that shows the cause-and-effect relationship between data, visualizing how each element affects other elements.

[0499] A "Bayesian network" is a type of probabilistic graphical model that refers to a network used to represent conditional dependencies between variables.

[0500] A "probability relationship" refers to the conditional probability relationship between variables, and is a mathematical expression of the degree to which each variable affects other variables.

[0501] A "specific event" refers to a specific abnormality or phenomenon of interest to the user (e.g., temperature abnormality, machine failure, etc.), and causal analysis is performed based on this.

[0502] "Influencing factors" refer to elements or variables that have an effect on a particular event or outcome, and include multiple factors that may be causative.

[0503] "Analytical means" refers to the methods and processes used to perform data analysis, particularly using statistical methods and machine learning algorithms.

[0504] The present invention relates to a system that effectively utilizes big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[0505] Data acquisition and preprocessing

[0506] The server captures IoT device data

[0507] The server collects data in real time from various IoT devices. This data includes sensor data such as temperature, pressure, and current. Specifically, the server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. This operation allows the data to be managed in a consistent format, making subsequent analysis easier.

[0508] The server completes missing data

[0509] Since the imported data often contains missing values, the server uses the fillna method from the Pandas library to fill in the missing values. Specifically, it applies "dataframe.fillna(method='ffill')" to the DataFrame to fill in the missing values ​​with the previous value.

[0510] Discovery of causal structures

[0511] The server discovers the causal structure

[0512] After preprocessing is complete, the server uses a generative AI model to discover causal relationships. This process utilizes machine learning libraries such as Scikit-learn and TensorFlow to learn causal relationships based on the features of the data. Specifically, it uses Linear Regression models and Granger Causality Tests to identify causal relationships between data.

[0513] Bayesian Network Training

[0514] The server builds a Bayesian network and learns probability relationships.

[0515] Based on the discovered causal structure, the server uses libraries such as PyMC3 and pgmpy to build a Bayesian network, which derives the conditional probability between each variable (node). Specifically, use "import pymc3 as pm" to define the model and calculate the conditional probability.

[0516] Causal analysis

[0517] The user specifies a specific event

[0518] A user sends a query about a specific anomaly or event (e.g., temperature anomaly) from their device to the server. This query includes details of the event they want to investigate. For example, a user might enter "Please analyze the cause of the temperature anomaly from the sensor data from Factory A" through a web or mobile application and submit it.

[0519] The server analyzes the causal relationships

[0520] Based on the query received from the user, the server analyzes the cause of the event using the pre-built Bayesian network. It calculates the influence from other related nodes and identifies the cause of the anomaly. Specifically, it applies "pm.sample_posterior_predictive(trace, model=model)" to calculate the probability distribution of the anomaly factors.

[0521] Returning results and making decisions

[0522] The server returns the analysis results to the device

[0523] Once the analysis is complete, the server returns the results to the user's device as a JSON-formatted HTTP response. Based on these analysis results, the user can consider appropriate maintenance plans and troubleshooting measures.

[0524] This invention enables the unified handling of big data in various formats held by companies and efficient data analysis, thereby enabling rapid and accurate decision-making regarding equipment maintenance and troubleshooting.

[0525] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0526] Step 1:

[0527] Data Ingestion

[0528] The server collects data in real time from various IoT devices. As input, it uses information from each IoT device's API endpoint and message queue (e.g., MQTT). The server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. The output is a CSV file in a unified format.

[0529] Specific behavior:

[0530] Get data from the IoT device using requests.get(api_endpoint).

[0531] Convert to CSV using data.to_csv('path / to / file.csv') and save.

[0532] Step 2:

[0533] Data Preprocessing

[0534] The server imputes missing values ​​in the imported data. The input is the CSV file obtained in step 1. The server uses the fillna method of the Pandas library to fill missing values ​​with the previous value. The output is a data frame with imputed missing values.

[0535] Specific behavior:

[0536] Read the CSV file with dataframe = pd.read_csv('path / to / file.csv').

[0537] Fill missing values ​​with dataframe.fillna(method='ffill').

[0538] Step 3:

[0539] Discovery of causal structures

[0540] Based on the preprocessed data, the server uses a generative AI model to discover the causal structure. The input is the data frame obtained in step 2. Causal relationships are learned using libraries such as Scikit-learn and TensorFlow. The output is a learned model that shows the causal relationships.

[0541] Specific behavior:

[0542] Define the model with model = LinearRegression().

[0543] Learn causal relationships with model.fit(X, y).

[0544] Step 4:

[0545] Bayesian Network Training

[0546] The server builds a Bayesian network based on the structural model of causal relationships and learns conditional probabilities. The input is the causal relationship model created in step 3. The Bayesian network is built using the PyMC3 and pgmpy libraries. The output is a Bayesian network probabilistic model.

[0547] Specific behavior:

[0548] Define the Bayesian network with pm.Model() as model:

[0549] Train a probabilistic model with pm.sample().

[0550] Step 5:

[0551] Receiving specific event analysis queries

[0552] The user sends a query about a specific anomaly or event (e.g., temperature anomaly) from a terminal to the server in the form of a prompt sentence. The input is the query text entered by the user. The output is the query sent to the server.

[0553] Specific behavior:

[0554] The user enters the following into the form on the terminal: "Please analyze the cause of the temperature abnormality from the sensor data of Factory A."

[0555] Click the submit button to send the query to the server.

[0556] Step 6:

[0557] Causal analysis

[0558] Based on the received query, the server analyzes the cause of the specified event using a Bayesian network. The inputs are the Bayesian network probabilistic model obtained in step 4 and the query received in step 5. The output is the analysis result of the anomaly cause.

[0559] Specific behavior:

[0560] Perform inference using pm.sample_posterior_predictive(trace, model=model).

[0561] Based on the query, determine the cause of the event.

[0562] Step 7:

[0563] Return of analysis results

[0564] The server returns the analysis results to the user's device. The input is the analysis results obtained in step 6. The output is the JSON data of the analysis results sent to the user.

[0565] Specific behavior:

[0566] Convert the parsed results into JSON format.

[0567] Return the response to the user device using response = HttpResponse(json.dumps(analysis result), content_type='application / json').

[0568] summary

[0569] In this way, the system automates the entire process from data collection to the return of analysis results, enabling efficient equipment management and troubleshooting.

[0570] (Application example 1)

[0571] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0572] There is a need to utilize big data in factory equipment to improve the efficiency of equipment maintenance and troubleshooting. However, currently, it is difficult to process and analyze huge amounts of data, and it is difficult to analyze missing data and causal relationships. Therefore, a system that can process data more efficiently and accurately and analyze anomalies is needed.

[0573] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0574] In this invention, the server includes a means for importing big data and supplementing missing data, a means for discovering a causal structure from the imported data using a learning model, a means for constructing a Bayesian network based on the learned model and learning probability relationships, a means for analyzing factors that influence specific events, a means for collecting sensor data from robots in a factory, and a means for identifying abnormalities in the robots based on the collected data and recommending a maintenance plan, thereby enabling efficient and accurate data processing and abnormality analysis.

[0575] "Big data" refers to the collection of huge amounts of data that are difficult to process and analyze using conventional methods.

[0576] "Missing data" is data that indicates a lack of some of the information available in a dataset.

[0577] A "learning model" is an algorithm that learns patterns and relationships from data and makes predictions and classifications.

[0578] "Causality" is a concept that indicates that one event has a cause-and-effect relationship with another event.

[0579] A "Bayesian network" is a model that expresses probabilistic causal relationships using a graph structure, and is a method for expressing relationships between variables based on conditional probability.

[0580] An "IoT device" is a physical device connected to the Internet, equipped with sensors and actuators and capable of collecting and transmitting data.

[0581] A "factory robot" is a machine designed to perform automated tasks in a factory environment.

[0582] "Sensor data" is data that expresses physical phenomena and environmental conditions measured by a sensor as numerical values.

[0583] A "generative AI model" is an AI model that uses machine learning algorithms to perform generative tasks (e.g., text generation or image generation).

[0584] A "maintenance plan" is a plan for regular inspections and repairs to keep facilities and equipment in good condition.

[0585] "Identifying anomalies" means detecting behavior or conditions that deviate from normal operating conditions and clarifying the cause.

[0586] The present invention provides a system for implementing efficient maintenance and troubleshooting by utilizing big data obtained from robots and equipment in a factory. Specific embodiments of the present invention will be described below.

[0587] 1. Data Collection and Preprocessing

[0588] Data collection

[0589] The server collects sensor data from IoT devices in the factory, including data on the equipment's status, such as temperature, pressure, current, and vibration, and transmits the data to the server in real time over the network.

[0590] Data Preprocessing

[0591] Since the collected data may contain missing values, the server preprocesses the data. Specifically, it uses the pandas library to convert the data into DataFrame format and imputes missing values. In this process, it uses forward fill, which fills in the previous value.

[0592] 2. Discovering causal structures

[0593] Discovering causal relationships using generative AI models

[0594] After the preprocessing is complete, the server uses a generative AI model to discover the causal relationships between the data. An example of a generative AI model is OpenAI's GPT model. A prompt is sent to this model to analyze the causal relationships.

[0595] Example prompt sentence:

[0596] "Please find the causal relationship of the abnormal occurrence of the factory robot based on the following data:

[0597] data:

[0598] {

[0599] "timestamp": ["2023-10-01 12:00:00", "2023-10-01 12:01:00", "2023-10-01 12:02:00"],

[0600] "temperature": [45, 47, 48],

[0601] "pressure": [101, 102, 100],

[0602] "current": [5.5, 5.6, 5.3]

[0603] }

[0604] "

[0605] 3. Building a Bayesian network

[0606] Building Bayesian networks based on structure discovery

[0607] Based on the generated causal relationships, the server constructs a Bayesian network, which models the probabilistic relationships between variables and describes the impact of certain events on others.

[0608] 4. Analyze abnormalities and recommend maintenance plans

[0609] Anomaly analysis

[0610] When a user specifies a specific abnormal event (e.g., abnormal temperature) on their smartphone, the server uses a Bayesian network to analyze the cause of the abnormality and identify the main factors.

[0611] Notification of results and recommended maintenance plans

[0612] The analysis results are sent to the user's smartphone, allowing them to plan and carry out maintenance in a timely manner.

[0613] Hardware and software used

[0614] Hardware

[0615] IoT devices (sensors, actuators, etc.)

[0616] Server (high performance computer)

[0617] Smartphone device

[0618] software

[0619] pandas library (data preprocessing)

[0620] OpenAI's generative AI model (discovering causal structures)

[0621] Bayesian network construction software

[0622] By implementing this invention, it is possible to efficiently process and analyze the huge amount of data obtained from robots and equipment in a factory, identify the cause of abnormalities, and create an appropriate maintenance plan, thereby reducing the risk of equipment failure and improving operational efficiency.

[0623] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0624] Step 1: Collect data

[0625] The server collects sensor data from IoT devices in the factory (for example, robots and their peripherals). Input data includes temperature, pressure, current, vibration, etc., and this data is sent to the server in real time via the network. Specifically, the server periodically requests data from each sensor and aggregates it.

[0626] Step 2: Preprocessing the data

[0627] The server converts the received raw data into DataFrame format using the pandas library. Since the data may contain missing values, it imputes the missing values. Specifically, the server applies forward fill to the missing parts of the dataframe to create a consistent dataset. The output is clean data with missing values ​​imputed.

[0628] Step 3: Discover the causal structure

[0629] The server uses the preprocessed data to send a prompt to the generative AI model (e.g., GPT model). The prompt contains the JSON formatted content of the sensor data. The input is the clean data and the prompt, which the server sends to the generative AI model. The output is structural information about the causal relationships.

[0630] Step 4: Building a Bayesian network

[0631] The server constructs a Bayesian network based on the structural information of causal relationships obtained from the generative AI model. Specifically, the server calculates the conditional probability between each variable and expresses this in a graph structure. The input is the structural information of causal relationships, and the output is the constructed Bayesian network.

[0632] Step 5: Analyze the anomaly

[0633] The user specifies a specific abnormal event (e.g., abnormal temperature) from their smartphone device and sends a query to the server. The server then uses a Bayesian network to analyze the input abnormal event. Specifically, the server analyzes the related nodes in the Bayesian network and identifies the cause of the abnormality. The input is the query from the user, and the output is the result of identifying the cause of the abnormality.

[0634] Step 6: Communicate results and recommend a maintenance plan

[0635] The server notifies the user's smartphone of the analysis results it has obtained. The notification includes the cause of the anomaly and specific maintenance instructions accordingly. Specifically, the server formats the analysis results as a text message and sends it to the user's device. The input is the anomaly analysis result, and the output is a notification message to the user.

[0636] This will enable real-time monitoring of robots and equipment within the factory, early detection of abnormalities, and appropriate maintenance measures.

[0637] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0638] The present invention is a system that combines a system for collecting big data, completing missing data, and analyzing causal relationships with an emotion engine that recognizes user emotions. Specific embodiments of the present invention will be described below.

[0639] 1. Data Acquisition and Preprocessing

[0640] The server collects IoT device data and emotion data.

[0641] The server takes in data from various IoT devices and user emotion data obtained through the emotion engine. This data is converted into a unified format (e.g., CSV file) and saved as a DataFrame that can be analyzed.

[0642] The server completes the missing data

[0643] If there are gaps in the imported data, the server will fill them in using an appropriate method, such as forward filling using the previous data value, to maintain data consistency.

[0644] 2. Discovering causal structures

[0645] The server discovers the causal structure

[0646] The server then uses generative AI models on the preprocessed data to discover causal structures, using machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[0647] 3. Building and training a Bayesian network

[0648] The server builds a Bayesian network and learns probability relationships.

[0649] The server builds a Bayesian network based on a structural model of causal relationships and learns the conditional probabilities between each piece of data. This learning process defines in detail the probabilistic relationships between each node.

[0650] 4. Causal Analysis

[0651] The user specifies a specific event

[0652] The user enters a query about a particular event of interest (e.g., a temperature anomaly or a particular emotional state) and sends it to the server. This query includes details of the event they want to investigate.

[0653] The server analyzes the causal relationships

[0654] Based on the query received, the server uses a Bayesian network to analyze the causal relationships of the specified events, specifically by identifying other variables that may affect a particular event and assessing the degree of their influence.

[0655] 5. Return of results and decision making

[0656] The server returns the analysis results to the device

[0657] Once the results of the causal analysis are obtained, the server sends the results back to the device, allowing the user to create appropriate maintenance plans and troubleshooting measures based on these results.

[0658] Specific examples

[0659] For example, suppose a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken:

[0660] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data obtained via the emotion engine, and converts the data into an analyzable format.

[0661] 2. The server imputes missing values ​​in the data: The server imputes missing values ​​in the dataset using previous values ​​to maintain data consistency.

[0662] 3. The server discovers causal structures: The server uses a generative AI model to analyze the causal relationships in the data and identify the causal relationships related to temperature anomalies and employees' emotional states (stress levels).

[0663] 4. The server constructs a Bayesian network and learns the probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between the data.

[0664] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[0665] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[0666] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[0667] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[0668] The processing flow will be explained below.

[0669] Step 1:

[0670] The server takes in IoT device data and user emotion data. Specifically, the server receives sensor data from factories and emotion data from wearable devices worn by employees. It converts this data into a unified format (e.g., CSV file) and saves it in an analyzable DataFrame.

[0671] Step 2:

[0672] The server completes missing data. Because the imported data is likely to contain missing values, the server completes the missing values ​​in an appropriate manner. Specifically, the server maintains data consistency by using methods such as forward filling with data values ​​from previous points in time.

[0673] Step 3:

[0674] The server discovers the causal structure. Using the preprocessed data as input, the server uses a generative AI model to discover the causal structure between each piece of data. This process uses machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[0675] Step 4:

[0676] The server builds a Bayesian network and learns the probabilistic relationships. Based on a structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. This learning process defines in detail the probabilistic relationships between each node (variable).

[0677] Step 5:

[0678] The user specifies a specific event. The user inputs a specific event of interest (e.g., a query about a temperature anomaly or a specific emotional state) from their device and sends it to the server. This query contains details of the event they want to investigate.

[0679] Step 6:

[0680] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies the influence of other variables related to the event and evaluates the degree of influence.

[0681] Step 7:

[0682] The server returns the analysis results to the device. Once the results of the causal relationship analysis are obtained, the server sends the results back to the device. Based on the analysis results, the user can create appropriate maintenance plans and troubleshooting measures.

[0683] Programmatic example

[0684] For example, if a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees, the user can do so as follows.

[0685] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data (e.g., stress level, heart rate, etc.) obtained via the emotion engine, and converts the data into an analyzable format.

[0686] 2. The server completes missing data: The server completes missing values ​​in the imported data with data values ​​from the previous point in time to maintain consistency.

[0687] 3. The server discovers causal structures: The server uses a generative AI model to analyze causal relationships between data and identify causal relationships related to temperature anomalies and employee stress levels.

[0688] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between data.

[0689] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[0690] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[0691] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[0692] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[0693] Example 2

[0694] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0695] Conventional big data analysis systems not only require a huge amount of time and effort to import and preprocess data and discover causal relationships, but also insufficiently utilize emotional data. Furthermore, the process for users to analyze causal relationships based on specific events and gain actionable insights is ineffective. This makes it difficult to make appropriate decisions and efficiently manage facilities.

[0696] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for importing big data and complementing missing data, means for discovering a causal structure from the imported data using a generative AI model, means for constructing a Bayesian network based on the learned model and learning probabilistic relationships, means for a user to input a query regarding a specific event and for the server to analyze the causal relationships of the event, means for returning the analysis results to the user's terminal, and means for importing and analyzing emotion data. This enables a user to efficiently integrate big data and emotion data, analyze causal relationships based on specific events, and obtain practical insights in real time.

[0697] "Big data" refers to a set of data that is so large and complex that it cannot be processed using conventional data processing technology.

[0698] "Missing data" refers to data in a dataset where some values ​​are missing or incomplete.

[0699] A "generative AI model" refers to an artificial intelligence model that is used to learn from data and generate new predictions and insights.

[0700] A "Bayesian network" refers to a graphical model for representing causal relationships between random variables and making inferences using conditional probabilities.

[0701] "Emotional data" refers to data that represents the user's emotional state (e.g., stress, satisfaction, fatigue, etc.).

[0702] A "query" is a question or request made to a database or system to retrieve or manipulate information.

[0703] "Causality" refers to the direct effect or relationship that a change in one variable has on another variable.

[0704] "Analysis" refers to the process of examining data in detail and uncovering hidden meanings and relationships within it.

[0705] "User" refers to a person or organization that uses the system.

[0706] "Terminal" refers to a device (e.g., PC, tablet, smartphone, etc.) that a user uses to access and operate the system.

[0707] The present invention relates to a system that collects big data, integrates user emotion data, and analyzes causal relationships. This system incorporates big data, complements missing data, discovers causal relationship structures, builds Bayesian networks, analyzes causal relationships based on specific events, returns analysis results, and analyzes emotion data. Specific embodiments of the present invention are described in detail below.

[0708] Data acquisition and preprocessing

[0709] The server collects IoT device data and emotion data.

[0710] The server collects various data using the factory's sensors (e.g., temperature sensors, humidity sensors, current sensors, etc.) and emotion engines (e.g., cameras and emotion recognition software). This data is converted to, for example, CSV format and saved as a DataFrame using Python's Pandas library. Specifically, the data is read using pd.read_csv('sensor_data.csv') and converted into a unified format.

[0711] The server converts the data into a unified format and stores it.

[0712] The collected data is converted into a unified format (e.g., date and time, sensor values), for example by unifying all timestamps to UTC and scaling data in different units. This converted data is saved as a DataFrame for further processing.

[0713] The server completes the missing data

[0714] Missing values ​​in the dataset are imputed using the Pandas fillna method, e.g., dataframe.fillna(method='ffill', inplace=True) , which imputes missing values ​​with the previous value and ensures data consistency.

[0715] Discovery of causal structures

[0716] The server uses a generative AI model to discover the causal structure

[0717] Once the preprocessing is complete, a generative AI model (e.g., a causal impact model) is used to discover the causal structure of the data. Specifically, a causal analysis algorithm is run as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period) .

[0718] Building and training Bayesian networks

[0719] The server builds a Bayesian network

[0720] Use the pgmpy library to build a Bayesian network based on a structural model. Define the network as follows: model = BayesianNetwork(structure) .

[0721] The server learns the probability relationships

[0722] A Bayesian network uses conditional probabilities to learn the probabilistic relationships between nodes. Specifically, it is trained using a data frame, such as model.fit(dataframe) .

[0723] Causal analysis and return of results

[0724] A user enters a query about a specific event

[0725] A user sends a query about a specific event (e.g., temperature abnormality or a specific emotional state) to the server via a terminal. For example, a user enters a query to investigate "employee stress levels when the temperature exceeds 30 degrees."

[0726] The server analyzes the causal relationships and returns the analysis results.

[0727] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. The analysis results are sent back to the user's device. Specifically, the server uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) .

[0728] Emotional data capture and analysis

[0729] The server captures and analyzes the emotional data.

[0730] Emotion data obtained from the emotion engine is collected in real time using IP cameras and emotion recognition software, which analyzes employees' emotional states (e.g., stress, satisfaction, fatigue) from their facial expressions and voice data.

[0731] Specific examples

[0732] For example, consider a case where a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken: The server imports sensor data (e.g., temperature, humidity, current, etc.) from each piece of factory equipment and employee emotion data obtained via an emotion engine. Then, it preprocesses the data and appropriately imputes missing data. Next, it uses a generative AI model to analyze the causal relationships in the data, constructs a Bayesian network, and learns the probabilistic relationships. The user sends a query to the server from their device to investigate the events "temperature abnormalities" and "employee stress levels," and the server analyzes the causal relationships using a Bayesian network. Finally, the analysis results are sent back to the user's device, allowing appropriate maintenance measures and employee care plans to be developed.

[0733] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0734] Step 1: Ingest your data

[0735] The server collects real-time data from various factory sensors (temperature sensors, humidity sensors, current sensors, etc.) and the emotion engine (camera and emotion recognition software). The input is the raw data from each sensor and the data obtained by the emotion engine, which is converted to CSV format and saved. The data is read as a DataFrame using the Pandas library, e.g., pd.read_csv('sensor_data.csv'). The output is a unified format DataFrame.

[0736] Step 2: Preprocessing the data

[0737] The server converts the read data into a unified format. Specifically, it converts all timestamps to UTC time and unifies units as necessary. The input is various sensor data and emotion data in CSV format, and it scales the data and converts timestamps. The output is a preprocessed DataFrame.

[0738] Step 3: Imputing missing data

[0739] The server uses Pandas' fillna method to fill missing values ​​in the dataset. Specifically, dataframe.fillna(method='ffill', inplace=True) fills missing parts with the previous value. The input is a preprocessed DataFrame, and the output is a DataFrame with missing data filled in.

[0740] Step 4: Discover the causal structure

[0741] The server uses the data after preprocessing and missing data imputation to discover the causal structure using a generative AI model. Specifically, it generates a model as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period). The input is a DataFrame with missing data imputed, and the output is a model showing the causal relationships.

[0742] Step 5: Building a Bayesian network

[0743] The server uses the pgmpy library to build a Bayesian network based on a causal model. Specifically, the network is defined as model = BayesianNetwork(structure) . The input is the causal model, and the output is the structure of the Bayesian network.

[0744] Step 6: Learning Probability Relationships

[0745] The server uses a Bayesian network to learn the conditional probabilities between each node. Specifically, training is performed using a data frame using model.fit(dataframe) . The input is a DataFrame with the Bayesian network structure and missing data imputed, and the output is a Bayesian network with learned conditional probabilities.

[0746] Step 7: Enter your query

[0747] A user uses a device to send a query about a specific event (e.g., temperature anomalies or a specific emotional state) to the server. The input is the user-specified event query, and the output is the data sent to the server for analysis of the query.

[0748] Step 8: Causal analysis

[0749] The server uses a Bayesian network to analyze the causal relationships of specified events based on the query submitted by the user. For example, it uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) . The input is the user's query and a Bayesian network trained on conditional probabilities, and the output is the analysis result.

[0750] Step 9: Returning the analysis results

[0751] The server returns the analysis results to the user's device. The user can use this information to formulate appropriate maintenance measures and employee care plans. The input is the analysis results, and the output is the analysis results returned to the user's device.

[0752] Specific prompt examples:

[0753] "Analyze the fluctuations in employees' stress levels when the temperature exceeds 30 degrees."

[0754] (Application example 2)

[0755] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0756] Conventional advertising systems were unable to generate optimal ads by comprehensively analyzing users' behavioral patterns and emotional states, resulting in limited advertising effectiveness. Furthermore, there were problems with reduced targeting accuracy when acquired data was incomplete or causal analysis was insufficient. Furthermore, the lack of a mechanism for evaluating advertising effectiveness in real time and dynamically feeding back the results limited the efficiency of the system.

[0757] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0758] In this invention, the server includes means for importing big data and supplementing missing data, means for discovering a causal structure from the imported data using a learning model, means for constructing a Bayesian network based on the learned model and learning probability relationships, means for collecting and analyzing user behavioral data and emotional data, means for generating and delivering optimal advertisements based on the user's behavioral patterns and emotional state using the Bayesian network, and means for evaluating the effectiveness of the delivered advertisements and feeding back the results to optimize the advertisement content. This enables highly accurate advertisement targeting and real-time evaluation based on the user's behavioral patterns and emotional state.

[0759] "Big data" refers to large, complex, and diverse data sets, as well as the technologies and methods for efficiently processing and analyzing them.

[0760] "Missing data" refers to a portion of a dataset where necessary information is missing, or the absence of a particular data point.

[0761] A "learning model" is a mathematical model that uses machine learning algorithms to learn patterns in data and make predictions or classifications.

[0762] A causal relationship is a relationship in which one event or variable influences another.

[0763] A "Bayesian network" is a type of probabilistic graphical model used to represent conditional dependencies between variables.

[0764] "User behavioral data" refers to data about a user's behavior when using a specific device or service, including, for example, location information and app usage history.

[0765] "Emotional data" refers to data that expresses a user's emotional state as numerical values ​​or categories, and is obtained by analyzing facial expressions, tone of voice, etc.

[0766] "Advertising" refers to information promoting a particular product or service, and refers to promotions or messages sent to users.

[0767] "Targeting" refers to the technology and methods of pinpointing and delivering the most appropriate advertisements and information to specific groups or individuals.

[0768] "Real-time evaluation" refers to the process of evaluating and analyzing specific events or data the moment they occur.

[0769] "Feedback" refers to information used to modify or optimize the next action or process based on the results obtained from an action or process that has already been performed.

[0770] MODE FOR CARRYING OUT THE INVENTION

[0771] The present invention provides a system that integrates and analyzes big data and user emotional data, and generates and delivers optimal advertisements based on the user's behavioral patterns and emotional state. This system includes the following means and processing steps.

[0772] System Configuration

[0773] Hardware and software configuration

[0774] 1. Hardware:

[0775] Smartphone: Collects user behavioral data (e.g., location information, app usage history) and emotional data through the camera.

[0776] Server: Preprocesses data, analyzes causal relationships, builds Bayesian networks, and generates and delivers advertisements.

[0777] 2. Software:

[0778] Data acquisition module: The data collected from the smartphone is acquired by the server.

[0779] Preprocessing module: Imputing missing data and data transformation.

[0780] Causal analysis module: Analyzes causal relationships between data using generative AI models.

[0781] Bayesian Network Module: Constructs and learns a Bayesian network based on the discovered causal relationships.

[0782] Advertisement generation and delivery module: Generates optimal advertisements based on the user's behavioral patterns and emotional state and delivers them to the smartphone.

[0783] Effectiveness evaluation module: Evaluates the effectiveness of delivered ads in real time and optimizes the ad content based on the results.

[0784] System Processing and Data Flow

[0785] 1. Data Acquisition and Preprocessing

[0786] The server collects location information, app usage history, and emotion data obtained from the camera in real time from the smartphone, converts it into a unified format (e.g., CSV file), and stores it as an analyzable DataFrame after completing a missing data imputation process.

[0787] 2. Causal analysis

[0788] The server uses the generative AI model on the data after data ingestion to identify causal relationships between the data. Specifically, it uses machine learning algorithms and statistical methods to discover hidden causal relationships.

[0789] 3. Building and training a Bayesian network

[0790] Based on the discovered causal structure, a Bayesian network is constructed to learn the conditional probabilities between each data point. This process defines the probabilistic relationships between variables in detail.

[0791] 4. Ad generation and delivery

[0792] If the server determines that the user is in a specific behavioral pattern or emotional state, it uses a Bayesian network to generate optimal advertisements and deliver them to the smartphone. For example, if the server determines that the user is having fun, it will deliver a coupon advertisement for a nearby cafe.

[0793] 5. Evaluation and feedback of advertising effectiveness

[0794] The effectiveness of delivered ads is evaluated in real time and ad content is optimized based on collected feedback data, thereby improving the accuracy of ad targeting.

[0795] Specific examples

[0796] For example, when a user is in a shopping mall, a smartphone app can analyze the user's emotional state through the camera. If the app determines that the user is having fun, it will deliver coupon advertisements for nearby cafes. If the user is tired, it will deliver advertisements for relaxation services.

[0797] Example prompt sentence:

[0798] "What kind of coupon ads can be delivered to users at the shopping mall to get the most impact when they are having fun?"

[0799] "When a user is tired in a shopping mall, what kind of relaxation service advertisement should be delivered to them to get the most effect?"

[0800] In this way, optimal advertisement delivery based on the user's emotional state is possible.

[0801] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0802] Step 1:

[0803] The server receives location information, app usage history, and emotional data obtained from the camera in real time from the smartphone. This data is converted into a standardized format (e.g., CSV file) and stored in a database.

[0804] Input: Location information from smartphone, app usage history, emotional data

[0805] Output: Data saved in a unified format (CSV file)

[0806] Step 2:

[0807] If there are missing parts in the imported data, the server will fill in the missing data by, for example, applying a forward fill method using the previous data value or a mean value interpolation method.

[0808] Input: Imported data, including missing data

[0809] Output: A complete and consistent dataset

[0810] Step 3:

[0811] The server analyzes the enriched dataset and uses a generative AI model to discover causal structures, a process that employs machine learning algorithms and statistical methods.

[0812] Input: Imputed dataset

[0813] Output: A structural model of the discovered causal relationships

[0814] Step 4:

[0815] The server constructs a Bayesian network based on a structural model of the discovered causal relationships and learns the conditional probabilities between each piece of data.

[0816] Input: structural model of causality

[0817] Output: Constructed Bayesian network and learned conditional probabilities

[0818] Step 5:

[0819] The server analyzes specific behavioral patterns and emotional states of users and generates optimal advertisements using a Bayesian network. It selects appropriate targeted advertisements based on the latest user data obtained from smartphones.

[0820] Input: User behavior patterns, emotional state, and Bayesian network

[0821] Output: The best ad generated

[0822] Step 6:

[0823] The server then delivers the generated advertisements to smartphones, adjusting the timing to ensure they are delivered at the appropriate time based on the user's current location and behavior.

[0824] Input: Generated ad, user's current location

[0825] Output: Advertisement delivered to the user's smartphone

[0826] Step 7:

[0827] The server evaluates the effectiveness of the delivered advertisements in real time and analyzes the collected feedback data. It collects user response data (click-through rate, duration of visit, etc.) and optimizes the advertisement content based on that data.

[0828] Input: Feedback data on advertising effectiveness

[0829] Output: Optimized advertising content and advertising strategies

[0830] This enables highly accurate ad delivery based on the user's behavioral patterns and emotional state, maximizing advertising effectiveness.

[0831] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0832] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0833] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0834] [Third embodiment]

[0835] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0836] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0837] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0838] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0839] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0840] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0841] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0842] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0843] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0844] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0845] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0846] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0847] The present invention provides a system for effectively utilizing big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[0848] 1. Data Acquisition and Preprocessing

[0849] The server captures IoT device data

[0850] The server ingests big data obtained from various IoT devices. This data usually comes in various formats and structures, so the server converts it into a unified format (e.g., CSV file) and ingests it into a DataFrame that can be analyzed.

[0851] The server completes the missing data

[0852] Since the ingested data is likely to contain missing values, the server performs preprocessing and imputes the missing values ​​in an appropriate way, for example, by using forward fill to maintain data consistency.

[0853] 2. Discovering causal structures

[0854] The server discovers the causal structure

[0855] After data preprocessing is complete, the server uses a generative AI model to discover the causal structure within the data. At this stage, advanced machine learning algorithms and statistical methods are used to uncover hidden causal relationships between data, and the resulting structural model is then used to build a Bayesian network.

[0856] 3. Bayesian Network Training

[0857] The server builds a Bayesian network and learns probability relationships.

[0858] The server builds a Bayesian network based on a structural model of causal relationships. The Bayesian network models the conditional probabilities between variables and establishes which variables influence which others. Through this learning process, the probabilistic relationships between each node (variable) are defined.

[0859] 4. Causal Analysis

[0860] The user specifies a specific event

[0861] When a user wants to analyze the causal relationship of a specific anomaly or event (e.g., temperature abnormality) in a piece of equipment, the user sends a query from the terminal to the server. This query includes details of the specific event they want to investigate.

[0862] The server analyzes the causal relationships

[0863] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. Specifically, it calculates the influence of other related nodes (variables) to identify the cause of the anomaly.

[0864] 5. Return of results and decision making

[0865] The server returns the analysis results to the device

[0866] Once the causal relationship analysis is complete, the server sends the results back to the device, allowing the user to create an appropriate maintenance plan or take appropriate measures to deal with problems.

[0867] Specific examples

[0868] For example, if a "temperature abnormality" occurs in a facility in a factory, the user uses the present invention to identify the cause. In this case, the following procedure is followed.

[0869] 1. The server collects IoT device data: The server collects sensor data (temperature, pressure, current, etc.) obtained from each piece of factory equipment and converts it into an analyzable format.

[0870] 2. The server completes missing data values: The server completes missing parts of the imported data to maintain data consistency.

[0871] 3. The server discovers the causal structure: The server uses a generative AI model to learn the causal relationships between the data and identify potential factors related to temperature anomalies.

[0872] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns a conditional probability table.

[0873] 5. The user specifies a specific event: The user sends an event called "temperature abnormality" from the terminal to the server as a query.

[0874] 6. Server analyzes causality: The server receives the query and analyzes the cause of the temperature anomaly using a Bayesian network.

[0875] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can then use the results to create an appropriate maintenance plan.

[0876] In this way, the present invention utilizes data held by a company to achieve efficient equipment management and troubleshooting.

[0877] The processing flow will be explained below.

[0878] Step 1:

[0879] The server imports IoT device data. Specifically, the server receives data obtained from various sensors and devices in a unified format such as CSV and converts it into an analyzable DataFrame. This data import is performed periodically, and data updates can also be made in real time.

[0880] Step 2:

[0881] The server completes missing data values. Because the imported data often contains missing values, the server performs preprocessing and completes the missing data in an appropriate manner. Specifically, it maintains data consistency by using techniques such as forward filling using previous data values.

[0882] Step 3:

[0883] The server discovers the causal structure. After preprocessing is complete, the server uses a generative AI model to explore causal relationships between the data. This involves using machine learning algorithms and statistical methods to generate a structural model that includes hidden causal relationships.

[0884] Step 4:

[0885] The server builds a Bayesian network and learns the probabilistic relationships. Based on the structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. In this learning process, the probabilistic relationships between each node are defined in detail.

[0886] Step 5:

[0887] The user specifies a specific event. The user enters a query about a specific event of interest (e.g., temperature anomaly) from the terminal and sends it to the server. This query contains details of the event they want to investigate.

[0888] Step 6:

[0889] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies other factors that may affect the event and evaluates the degree of influence.

[0890] Step 7:

[0891] The server returns the analysis results to the terminal. Once the causal relationship analysis results are obtained, the server sends the results back to the terminal. Based on these results, the user can create equipment maintenance plans and troubleshoot problems.

[0892] The above are the specific processing steps of the system of the present invention, which enable companies to effectively utilize big data and optimize their asset management through advanced causal analysis.

[0893] Example 1

[0894] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0895] Conventional systems make it difficult to efficiently utilize a company's big data and effectively carry out equipment maintenance and troubleshooting. In particular, converting data into a unified format, filling in missing data, and automatically discovering and analyzing causal relationships are often done manually, which is time-consuming and costly. It is also difficult to visualize the structure of causal relationships and quickly identify the factors that affect specific events. There is a need for a system that can solve this issue and improve the efficiency of equipment management and troubleshooting.

[0896] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0897] In this invention, the server includes means for importing big data and supplementing missing data, means for converting the imported data into a unified format and processing it into an analyzable structure, means for discovering a causal structure from preprocessed data using a generative AI model, means for building a Bayesian network based on the learned causal structure and learning probability relationships, and means for analyzing factors that influence specific events. This enables unified handling of big data in various formats held by a company, efficient data analysis, and rapid decision-making for equipment maintenance and troubleshooting.

[0898] "Big data" refers to large amounts of diverse, rapidly generated data sets that are difficult to process using traditional database management tools.

[0899] "Missing data" refers to values ​​that are missing from a dataset for some reason, and is often expressed in the form of NaN (Not a Number) or similar.

[0900] "Uniform format" refers to converting data from various formats into a consistent format (e.g., CSV, DataFrame, etc.), which makes data analysis easier.

[0901] "Analyzable structure" means that the data has a format and structure suitable for analysis, particularly a data frame or table format.

[0902] "Generative AI model" refers to a learning model generated using artificial intelligence, specifically including models built using machine learning or deep learning algorithms.

[0903] A "causal structure" refers to a model that shows the cause-and-effect relationship between data, visualizing how each element affects other elements.

[0904] A "Bayesian network" is a type of probabilistic graphical model that refers to a network used to represent conditional dependencies between variables.

[0905] A "probability relationship" refers to the conditional probability relationship between variables, and is a mathematical expression of the degree to which each variable affects other variables.

[0906] A "specific event" refers to a specific abnormality or phenomenon of interest to the user (e.g., temperature abnormality, machine failure, etc.), and causal analysis is performed based on this.

[0907] "Influencing factors" refer to elements or variables that have an effect on a particular event or outcome, and include multiple factors that may be causative.

[0908] "Analytical means" refers to the methods and processes used to perform data analysis, particularly using statistical methods and machine learning algorithms.

[0909] The present invention relates to a system that effectively utilizes big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[0910] Data acquisition and preprocessing

[0911] The server captures IoT device data

[0912] The server collects data in real time from various IoT devices. This data includes sensor data such as temperature, pressure, and current. Specifically, the server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. This operation allows the data to be managed in a consistent format, making subsequent analysis easier.

[0913] The server completes missing data

[0914] Since the imported data often contains missing values, the server uses the fillna method from the Pandas library to fill in the missing values. Specifically, it applies "dataframe.fillna(method='ffill')" to the DataFrame to fill in the missing values ​​with the previous value.

[0915] Discovery of causal structures

[0916] The server discovers the causal structure

[0917] After preprocessing is complete, the server uses a generative AI model to discover causal relationships. This process utilizes machine learning libraries such as Scikit-learn and TensorFlow to learn causal relationships based on the features of the data. Specifically, it uses Linear Regression models and Granger Causality Tests to identify causal relationships between data.

[0918] Bayesian Network Training

[0919] The server builds a Bayesian network and learns probability relationships.

[0920] Based on the discovered causal structure, the server uses libraries such as PyMC3 and pgmpy to build a Bayesian network, which derives the conditional probability between each variable (node). Specifically, use "import pymc3 as pm" to define the model and calculate the conditional probability.

[0921] Causal analysis

[0922] The user specifies a specific event

[0923] A user sends a query about a specific anomaly or event (e.g., temperature anomaly) from their device to the server. This query includes details of the event they want to investigate. For example, a user might enter "Please analyze the cause of the temperature anomaly from the sensor data from Factory A" through a web or mobile application and submit it.

[0924] The server analyzes the causal relationships

[0925] Based on the query received from the user, the server analyzes the cause of the event using the pre-built Bayesian network. It calculates the influence from other related nodes and identifies the cause of the anomaly. Specifically, it applies "pm.sample_posterior_predictive(trace, model=model)" to calculate the probability distribution of the anomaly factors.

[0926] Returning results and making decisions

[0927] The server returns the analysis results to the device

[0928] Once the analysis is complete, the server returns the results to the user's device as a JSON-formatted HTTP response. Based on these analysis results, the user can consider appropriate maintenance plans and troubleshooting measures.

[0929] This invention enables the unified handling of big data in various formats held by companies and efficient data analysis, thereby enabling rapid and accurate decision-making regarding equipment maintenance and troubleshooting.

[0930] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0931] Step 1:

[0932] Data Ingestion

[0933] The server collects data in real time from various IoT devices. As input, it uses information from each IoT device's API endpoint and message queue (e.g., MQTT). The server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. The output is a CSV file in a unified format.

[0934] Specific behavior:

[0935] Get data from the IoT device using requests.get(api_endpoint).

[0936] Convert to CSV using data.to_csv('path / to / file.csv') and save.

[0937] Step 2:

[0938] Data Preprocessing

[0939] The server imputes missing values ​​in the imported data. The input is the CSV file obtained in step 1. The server uses the fillna method of the Pandas library to fill missing values ​​with the previous value. The output is a data frame with imputed missing values.

[0940] Specific behavior:

[0941] Read the CSV file with dataframe = pd.read_csv('path / to / file.csv').

[0942] Fill missing values ​​with dataframe.fillna(method='ffill').

[0943] Step 3:

[0944] Discovery of causal structures

[0945] Based on the preprocessed data, the server uses a generative AI model to discover the causal structure. The input is the data frame obtained in step 2. Causal relationships are learned using libraries such as Scikit-learn and TensorFlow. The output is a learned model that shows the causal relationships.

[0946] Specific behavior:

[0947] Define the model with model = LinearRegression().

[0948] Learn causal relationships with model.fit(X, y).

[0949] Step 4:

[0950] Bayesian Network Training

[0951] The server builds a Bayesian network based on the structural model of causal relationships and learns conditional probabilities. The input is the causal relationship model created in step 3. The Bayesian network is built using the PyMC3 and pgmpy libraries. The output is a Bayesian network probabilistic model.

[0952] Specific behavior:

[0953] Define the Bayesian network with pm.Model() as model:

[0954] Train a probabilistic model with pm.sample().

[0955] Step 5:

[0956] Receiving specific event analysis queries

[0957] The user sends a query about a specific anomaly or event (e.g., temperature anomaly) from a terminal to the server in the form of a prompt sentence. The input is the query text entered by the user. The output is the query sent to the server.

[0958] Specific behavior:

[0959] The user enters the following into the form on the terminal: "Please analyze the cause of the temperature abnormality from the sensor data of Factory A."

[0960] Click the submit button to send the query to the server.

[0961] Step 6:

[0962] Causal analysis

[0963] Based on the received query, the server analyzes the cause of the specified event using a Bayesian network. The inputs are the Bayesian network probabilistic model obtained in step 4 and the query received in step 5. The output is the analysis result of the anomaly cause.

[0964] Specific behavior:

[0965] Perform inference using pm.sample_posterior_predictive(trace, model=model).

[0966] Based on the query, determine the cause of the event.

[0967] Step 7:

[0968] Return of analysis results

[0969] The server returns the analysis results to the user's device. The input is the analysis results obtained in step 6. The output is the JSON data of the analysis results sent to the user.

[0970] Specific behavior:

[0971] Convert the parsed results into JSON format.

[0972] Return the response to the user device using response = HttpResponse(json.dumps(analysis result), content_type='application / json').

[0973] summary

[0974] In this way, the system automates the entire process from data collection to the return of analysis results, enabling efficient equipment management and troubleshooting.

[0975] (Application example 1)

[0976] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0977] There is a need to utilize big data in factory equipment to improve the efficiency of equipment maintenance and troubleshooting. However, currently, it is difficult to process and analyze huge amounts of data, and it is difficult to analyze missing data and causal relationships. Therefore, a system that can process data more efficiently and accurately and analyze anomalies is needed.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0979] In this invention, the server includes a means for importing big data and supplementing missing data, a means for discovering a causal structure from the imported data using a learning model, a means for constructing a Bayesian network based on the learned model and learning probability relationships, a means for analyzing factors that influence specific events, a means for collecting sensor data from robots in a factory, and a means for identifying abnormalities in the robots based on the collected data and recommending a maintenance plan, thereby enabling efficient and accurate data processing and abnormality analysis.

[0980] "Big data" refers to the collection of huge amounts of data that are difficult to process and analyze using conventional methods.

[0981] "Missing data" is data that indicates a lack of some of the information available in a dataset.

[0982] A "learning model" is an algorithm that learns patterns and relationships from data and makes predictions and classifications.

[0983] "Causality" is a concept that indicates that one event has a cause-and-effect relationship with another event.

[0984] A "Bayesian network" is a model that expresses probabilistic causal relationships using a graph structure, and is a method for expressing relationships between variables based on conditional probability.

[0985] An "IoT device" is a physical device connected to the Internet, equipped with sensors and actuators and capable of collecting and transmitting data.

[0986] A "factory robot" is a machine designed to perform automated tasks in a factory environment.

[0987] "Sensor data" is data that expresses physical phenomena and environmental conditions measured by a sensor as numerical values.

[0988] A "generative AI model" is an AI model that uses machine learning algorithms to perform generative tasks (e.g., text generation or image generation).

[0989] A "maintenance plan" is a plan for regular inspections and repairs to keep facilities and equipment in good condition.

[0990] "Identifying anomalies" means detecting behavior or conditions that deviate from normal operating conditions and clarifying the cause.

[0991] The present invention provides a system for implementing efficient maintenance and troubleshooting by utilizing big data obtained from robots and equipment in a factory. Specific embodiments of the present invention will be described below.

[0992] 1. Data Collection and Preprocessing

[0993] Data collection

[0994] The server collects sensor data from IoT devices in the factory, including data on the equipment's status, such as temperature, pressure, current, and vibration, and transmits the data to the server in real time over the network.

[0995] Data Preprocessing

[0996] Since the collected data may contain missing values, the server preprocesses the data. Specifically, it uses the pandas library to convert the data into DataFrame format and imputes missing values. In this process, it uses forward fill, which fills in the previous value.

[0997] 2. Discovering causal structures

[0998] Discovering causal relationships using generative AI models

[0999] After the preprocessing is complete, the server uses a generative AI model to discover the causal relationships between the data. An example of a generative AI model is OpenAI's GPT model. A prompt is sent to this model to analyze the causal relationships.

[1000] Example prompt sentence:

[1001] "Please find the causal relationship of the abnormal occurrence of the factory robot based on the following data:

[1002] data:

[1003] {

[1004] "timestamp": ["2023-10-01 12:00:00", "2023-10-01 12:01:00", "2023-10-01 12:02:00"],

[1005] "temperature": [45, 47, 48],

[1006] "pressure": [101, 102, 100],

[1007] "current": [5.5, 5.6, 5.3]

[1008] }

[1009] "

[1010] 3. Building a Bayesian network

[1011] Building Bayesian networks based on structure discovery

[1012] Based on the generated causal relationships, the server constructs a Bayesian network, which models the probabilistic relationships between variables and describes the impact of certain events on others.

[1013] 4. Analyze abnormalities and recommend maintenance plans

[1014] Anomaly analysis

[1015] When a user specifies a specific abnormal event (e.g., abnormal temperature) on their smartphone, the server uses a Bayesian network to analyze the cause of the abnormality and identify the main factors.

[1016] Notification of results and recommended maintenance plans

[1017] The analysis results are sent to the user's smartphone, allowing them to plan and carry out maintenance in a timely manner.

[1018] Hardware and software used

[1019] Hardware

[1020] IoT devices (sensors, actuators, etc.)

[1021] Server (high performance computer)

[1022] Smartphone device

[1023] software

[1024] pandas library (data preprocessing)

[1025] OpenAI's generative AI model (discovering causal structures)

[1026] Bayesian network construction software

[1027] By implementing this invention, it is possible to efficiently process and analyze the huge amount of data obtained from robots and equipment in a factory, identify the cause of abnormalities, and create an appropriate maintenance plan, thereby reducing the risk of equipment failure and improving operational efficiency.

[1028] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1029] Step 1: Collect data

[1030] The server collects sensor data from IoT devices in the factory (for example, robots and their peripherals). Input data includes temperature, pressure, current, vibration, etc., and this data is sent to the server in real time via the network. Specifically, the server periodically requests data from each sensor and aggregates it.

[1031] Step 2: Preprocessing the data

[1032] The server converts the received raw data into DataFrame format using the pandas library. Since the data may contain missing values, it imputes the missing values. Specifically, the server applies forward fill to the missing parts of the dataframe to create a consistent dataset. The output is clean data with missing values ​​imputed.

[1033] Step 3: Discover the causal structure

[1034] The server uses the preprocessed data to send a prompt to the generative AI model (e.g., GPT model). The prompt contains the JSON formatted content of the sensor data. The input is the clean data and the prompt, which the server sends to the generative AI model. The output is structural information about the causal relationships.

[1035] Step 4: Building a Bayesian network

[1036] The server constructs a Bayesian network based on the structural information of causal relationships obtained from the generative AI model. Specifically, the server calculates the conditional probability between each variable and expresses this in a graph structure. The input is the structural information of causal relationships, and the output is the constructed Bayesian network.

[1037] Step 5: Analyze the anomaly

[1038] The user specifies a specific abnormal event (e.g., abnormal temperature) from their smartphone device and sends a query to the server. The server then uses a Bayesian network to analyze the input abnormal event. Specifically, the server analyzes the related nodes in the Bayesian network and identifies the cause of the abnormality. The input is the query from the user, and the output is the result of identifying the cause of the abnormality.

[1039] Step 6: Communicate results and recommend a maintenance plan

[1040] The server notifies the user's smartphone of the analysis results it has obtained. The notification includes the cause of the anomaly and specific maintenance instructions accordingly. Specifically, the server formats the analysis results as a text message and sends it to the user's device. The input is the anomaly analysis result, and the output is a notification message to the user.

[1041] This will enable real-time monitoring of robots and equipment within the factory, early detection of abnormalities, and appropriate maintenance measures.

[1042] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1043] The present invention is a system that combines a system for collecting big data, completing missing data, and analyzing causal relationships with an emotion engine that recognizes user emotions. Specific embodiments of the present invention will be described below.

[1044] 1. Data Acquisition and Preprocessing

[1045] The server collects IoT device data and emotion data.

[1046] The server takes in data from various IoT devices and user emotion data obtained through the emotion engine. This data is converted into a unified format (e.g., CSV file) and saved as a DataFrame that can be analyzed.

[1047] The server completes the missing data

[1048] If there are gaps in the imported data, the server will fill them in using an appropriate method, such as forward filling using the previous data value, to maintain data consistency.

[1049] 2. Discovering causal structures

[1050] The server discovers the causal structure

[1051] The server then uses generative AI models on the preprocessed data to discover causal structures, using machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[1052] 3. Building and training a Bayesian network

[1053] The server builds a Bayesian network and learns probability relationships.

[1054] The server builds a Bayesian network based on a structural model of causal relationships and learns the conditional probabilities between each piece of data. This learning process defines in detail the probabilistic relationships between each node.

[1055] 4. Causal Analysis

[1056] The user specifies a specific event

[1057] The user enters a query about a particular event of interest (e.g., a temperature anomaly or a particular emotional state) and sends it to the server. This query includes details of the event they want to investigate.

[1058] The server analyzes the causal relationships

[1059] Based on the query received, the server uses a Bayesian network to analyze the causal relationships of the specified events, specifically by identifying other variables that may affect a particular event and assessing the degree of their influence.

[1060] 5. Return of results and decision making

[1061] The server returns the analysis results to the device

[1062] Once the results of the causal analysis are obtained, the server sends the results back to the device, allowing the user to create appropriate maintenance plans and troubleshooting measures based on these results.

[1063] Specific examples

[1064] For example, suppose a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken:

[1065] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data obtained via the emotion engine, and converts the data into an analyzable format.

[1066] 2. The server imputes missing values ​​in the data: The server imputes missing values ​​in the dataset using previous values ​​to maintain data consistency.

[1067] 3. The server discovers causal structures: The server uses a generative AI model to analyze the causal relationships in the data and identify the causal relationships related to temperature anomalies and employees' emotional states (stress levels).

[1068] 4. The server constructs a Bayesian network and learns the probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between the data.

[1069] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[1070] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[1071] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[1072] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[1073] The processing flow will be explained below.

[1074] Step 1:

[1075] The server takes in IoT device data and user emotion data. Specifically, the server receives sensor data from factories and emotion data from wearable devices worn by employees. It converts this data into a unified format (e.g., CSV file) and saves it in an analyzable DataFrame.

[1076] Step 2:

[1077] The server completes missing data. Because the imported data is likely to contain missing values, the server completes the missing values ​​in an appropriate manner. Specifically, the server maintains data consistency by using methods such as forward filling with data values ​​from previous points in time.

[1078] Step 3:

[1079] The server discovers the causal structure. Using the preprocessed data as input, the server uses a generative AI model to discover the causal structure between each piece of data. This process uses machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[1080] Step 4:

[1081] The server builds a Bayesian network and learns the probabilistic relationships. Based on a structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. This learning process defines in detail the probabilistic relationships between each node (variable).

[1082] Step 5:

[1083] The user specifies a specific event. The user inputs a specific event of interest (e.g., a query about a temperature anomaly or a specific emotional state) from their device and sends it to the server. This query contains details of the event they want to investigate.

[1084] Step 6:

[1085] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies the influence of other variables related to the event and evaluates the degree of influence.

[1086] Step 7:

[1087] The server returns the analysis results to the device. Once the results of the causal relationship analysis are obtained, the server sends the results back to the device. Based on the analysis results, the user can create appropriate maintenance plans and troubleshooting measures.

[1088] Programmatic example

[1089] For example, if a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees, the user can do so as follows.

[1090] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data (e.g., stress level, heart rate, etc.) obtained via the emotion engine, and converts the data into an analyzable format.

[1091] 2. The server completes missing data: The server completes missing values ​​in the imported data with data values ​​from the previous point in time to maintain consistency.

[1092] 3. The server discovers causal structures: The server uses a generative AI model to analyze causal relationships between data and identify causal relationships related to temperature anomalies and employee stress levels.

[1093] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between data.

[1094] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[1095] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[1096] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[1097] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[1098] Example 2

[1099] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1100] Conventional big data analysis systems not only require a huge amount of time and effort to import and preprocess data and discover causal relationships, but also insufficiently utilize emotional data. Furthermore, the process for users to analyze causal relationships based on specific events and gain actionable insights is ineffective. This makes it difficult to make appropriate decisions and efficiently manage facilities.

[1101] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for importing big data and complementing missing data, means for discovering a causal structure from the imported data using a generative AI model, means for constructing a Bayesian network based on the learned model and learning probabilistic relationships, means for a user to input a query regarding a specific event and for the server to analyze the causal relationships of the event, means for returning the analysis results to the user's terminal, and means for importing and analyzing emotion data. This enables a user to efficiently integrate big data and emotion data, analyze causal relationships based on specific events, and obtain practical insights in real time.

[1102] "Big data" refers to a set of data that is so large and complex that it cannot be processed using conventional data processing technology.

[1103] "Missing data" refers to data in a dataset where some values ​​are missing or incomplete.

[1104] A "generative AI model" refers to an artificial intelligence model that is used to learn from data and generate new predictions and insights.

[1105] A "Bayesian network" refers to a graphical model for representing causal relationships between random variables and making inferences using conditional probabilities.

[1106] "Emotional data" refers to data that represents the user's emotional state (e.g., stress, satisfaction, fatigue, etc.).

[1107] A "query" is a question or request made to a database or system to retrieve or manipulate information.

[1108] "Causality" refers to the direct effect or relationship that a change in one variable has on another variable.

[1109] "Analysis" refers to the process of examining data in detail and uncovering hidden meanings and relationships within it.

[1110] "User" refers to a person or organization that uses the system.

[1111] "Terminal" refers to a device (e.g., PC, tablet, smartphone, etc.) that a user uses to access and operate the system.

[1112] The present invention relates to a system that collects big data, integrates user emotion data, and analyzes causal relationships. This system incorporates big data, complements missing data, discovers causal relationship structures, builds Bayesian networks, analyzes causal relationships based on specific events, returns analysis results, and analyzes emotion data. Specific embodiments of the present invention are described in detail below.

[1113] Data acquisition and preprocessing

[1114] The server collects IoT device data and emotion data.

[1115] The server collects various data using the factory's sensors (e.g., temperature sensors, humidity sensors, current sensors, etc.) and emotion engines (e.g., cameras and emotion recognition software). This data is converted to, for example, CSV format and saved as a DataFrame using Python's Pandas library. Specifically, the data is read using pd.read_csv('sensor_data.csv') and converted into a unified format.

[1116] The server converts the data into a unified format and stores it.

[1117] The collected data is converted into a unified format (e.g., date and time, sensor values), for example by unifying all timestamps to UTC and scaling data in different units. This converted data is saved as a DataFrame for further processing.

[1118] The server completes the missing data

[1119] Missing values ​​in the dataset are imputed using the Pandas fillna method, e.g., dataframe.fillna(method='ffill', inplace=True) , which imputes missing values ​​with the previous value and ensures data consistency.

[1120] Discovery of causal structures

[1121] The server uses a generative AI model to discover the causal structure

[1122] Once the preprocessing is complete, a generative AI model (e.g., a causal impact model) is used to discover the causal structure of the data. Specifically, a causal analysis algorithm is run as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period) .

[1123] Building and training Bayesian networks

[1124] The server builds a Bayesian network

[1125] Use the pgmpy library to build a Bayesian network based on a structural model. Define the network as follows: model = BayesianNetwork(structure) .

[1126] The server learns the probability relationships

[1127] A Bayesian network uses conditional probabilities to learn the probabilistic relationships between nodes. Specifically, it is trained using a data frame, such as model.fit(dataframe) .

[1128] Causal analysis and return of results

[1129] A user enters a query about a specific event

[1130] A user sends a query about a specific event (e.g., temperature abnormality or a specific emotional state) to the server via a terminal. For example, a user enters a query to investigate "employee stress levels when the temperature exceeds 30 degrees."

[1131] The server analyzes the causal relationships and returns the analysis results.

[1132] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. The analysis results are sent back to the user's device. Specifically, the server uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) .

[1133] Emotional data capture and analysis

[1134] The server captures and analyzes the emotional data.

[1135] Emotion data obtained from the emotion engine is collected in real time using IP cameras and emotion recognition software, which analyzes employees' emotional states (e.g., stress, satisfaction, fatigue) from their facial expressions and voice data.

[1136] Specific examples

[1137] For example, consider a case where a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken: The server imports sensor data (e.g., temperature, humidity, current, etc.) from each piece of factory equipment and employee emotion data obtained via an emotion engine. Then, it preprocesses the data and appropriately imputes missing data. Next, it uses a generative AI model to analyze the causal relationships in the data, constructs a Bayesian network, and learns the probabilistic relationships. The user sends a query to the server from their device to investigate the events "temperature abnormalities" and "employee stress levels," and the server analyzes the causal relationships using a Bayesian network. Finally, the analysis results are sent back to the user's device, allowing appropriate maintenance measures and employee care plans to be developed.

[1138] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1139] Step 1: Ingest your data

[1140] The server collects real-time data from various factory sensors (temperature sensors, humidity sensors, current sensors, etc.) and the emotion engine (camera and emotion recognition software). The input is the raw data from each sensor and the data obtained by the emotion engine, which is converted to CSV format and saved. The data is read as a DataFrame using the Pandas library, e.g., pd.read_csv('sensor_data.csv'). The output is a unified format DataFrame.

[1141] Step 2: Preprocessing the data

[1142] The server converts the read data into a unified format. Specifically, it converts all timestamps to UTC time and unifies units as necessary. The input is various sensor data and emotion data in CSV format, and it scales the data and converts timestamps. The output is a preprocessed DataFrame.

[1143] Step 3: Imputing missing data

[1144] The server uses Pandas' fillna method to fill missing values ​​in the dataset. Specifically, dataframe.fillna(method='ffill', inplace=True) fills missing parts with the previous value. The input is a preprocessed DataFrame, and the output is a DataFrame with missing data filled in.

[1145] Step 4: Discover the causal structure

[1146] The server uses the data after preprocessing and missing data imputation to discover the causal structure using a generative AI model. Specifically, it generates a model as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period). The input is a DataFrame with missing data imputed, and the output is a model showing the causal relationships.

[1147] Step 5: Building a Bayesian network

[1148] The server uses the pgmpy library to build a Bayesian network based on a causal model. Specifically, the network is defined as model = BayesianNetwork(structure) . The input is the causal model, and the output is the structure of the Bayesian network.

[1149] Step 6: Learning Probability Relationships

[1150] The server uses a Bayesian network to learn the conditional probabilities between each node. Specifically, training is performed using a data frame using model.fit(dataframe) . The input is a DataFrame with the Bayesian network structure and missing data imputed, and the output is a Bayesian network with learned conditional probabilities.

[1151] Step 7: Enter your query

[1152] A user uses a device to send a query about a specific event (e.g., temperature anomalies or a specific emotional state) to the server. The input is the user-specified event query, and the output is the data sent to the server for analysis of the query.

[1153] Step 8: Causal analysis

[1154] The server uses a Bayesian network to analyze the causal relationships of specified events based on the query submitted by the user. For example, it uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) . The input is the user's query and a Bayesian network trained on conditional probabilities, and the output is the analysis result.

[1155] Step 9: Returning the analysis results

[1156] The server returns the analysis results to the user's device. The user can use this information to formulate appropriate maintenance measures and employee care plans. The input is the analysis results, and the output is the analysis results returned to the user's device.

[1157] Specific prompt examples:

[1158] "Analyze the fluctuations in employees' stress levels when the temperature exceeds 30 degrees."

[1159] (Application example 2)

[1160] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1161] Conventional advertising systems were unable to generate optimal ads by comprehensively analyzing users' behavioral patterns and emotional states, resulting in limited advertising effectiveness. Furthermore, there were problems with reduced targeting accuracy when acquired data was incomplete or causal analysis was insufficient. Furthermore, the lack of a mechanism for evaluating advertising effectiveness in real time and dynamically feeding back the results limited the efficiency of the system.

[1162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1163] In this invention, the server includes means for importing big data and supplementing missing data, means for discovering a causal structure from the imported data using a learning model, means for constructing a Bayesian network based on the learned model and learning probability relationships, means for collecting and analyzing user behavioral data and emotional data, means for generating and delivering optimal advertisements based on the user's behavioral patterns and emotional state using the Bayesian network, and means for evaluating the effectiveness of the delivered advertisements and feeding back the results to optimize the advertisement content. This enables highly accurate advertisement targeting and real-time evaluation based on the user's behavioral patterns and emotional state.

[1164] "Big data" refers to large, complex, and diverse data sets, as well as the technologies and methods for efficiently processing and analyzing them.

[1165] "Missing data" refers to a portion of a dataset where necessary information is missing, or the absence of a particular data point.

[1166] A "learning model" is a mathematical model that uses machine learning algorithms to learn patterns in data and make predictions or classifications.

[1167] A causal relationship is a relationship in which one event or variable influences another.

[1168] A "Bayesian network" is a type of probabilistic graphical model used to represent conditional dependencies between variables.

[1169] "User behavioral data" refers to data about a user's behavior when using a specific device or service, including, for example, location information and app usage history.

[1170] "Emotional data" refers to data that expresses a user's emotional state as numerical values ​​or categories, and is obtained by analyzing facial expressions, tone of voice, etc.

[1171] "Advertising" refers to information promoting a particular product or service, and refers to promotions or messages sent to users.

[1172] "Targeting" refers to the technology and methods of pinpointing and delivering the most appropriate advertisements and information to specific groups or individuals.

[1173] "Real-time evaluation" refers to the process of evaluating and analyzing specific events or data the moment they occur.

[1174] "Feedback" refers to information used to modify or optimize the next action or process based on the results obtained from an action or process that has already been performed.

[1175] MODE FOR CARRYING OUT THE INVENTION

[1176] The present invention provides a system that integrates and analyzes big data and user emotional data, and generates and delivers optimal advertisements based on the user's behavioral patterns and emotional state. This system includes the following means and processing steps.

[1177] System Configuration

[1178] Hardware and software configuration

[1179] 1. Hardware:

[1180] Smartphone: Collects user behavioral data (e.g., location information, app usage history) and emotional data through the camera.

[1181] Server: Preprocesses data, analyzes causal relationships, builds Bayesian networks, and generates and delivers advertisements.

[1182] 2. Software:

[1183] Data acquisition module: The data collected from the smartphone is acquired by the server.

[1184] Preprocessing module: Imputing missing data and data transformation.

[1185] Causal analysis module: Analyzes causal relationships between data using generative AI models.

[1186] Bayesian Network Module: Constructs and learns a Bayesian network based on the discovered causal relationships.

[1187] Advertisement generation and delivery module: Generates optimal advertisements based on the user's behavioral patterns and emotional state and delivers them to the smartphone.

[1188] Effectiveness evaluation module: Evaluates the effectiveness of delivered ads in real time and optimizes the ad content based on the results.

[1189] System Processing and Data Flow

[1190] 1. Data Acquisition and Preprocessing

[1191] The server collects location information, app usage history, and emotion data obtained from the camera in real time from the smartphone, converts it into a unified format (e.g., CSV file), and stores it as an analyzable DataFrame after completing a missing data imputation process.

[1192] 2. Causal analysis

[1193] The server uses the generative AI model on the data after data ingestion to identify causal relationships between the data. Specifically, it uses machine learning algorithms and statistical methods to discover hidden causal relationships.

[1194] 3. Building and training a Bayesian network

[1195] Based on the discovered causal structure, a Bayesian network is constructed to learn the conditional probabilities between each data point. This process defines the probabilistic relationships between variables in detail.

[1196] 4. Ad generation and delivery

[1197] If the server determines that the user is in a specific behavioral pattern or emotional state, it uses a Bayesian network to generate optimal advertisements and deliver them to the smartphone. For example, if the server determines that the user is having fun, it will deliver a coupon advertisement for a nearby cafe.

[1198] 5. Evaluation and feedback of advertising effectiveness

[1199] The effectiveness of delivered ads is evaluated in real time and ad content is optimized based on collected feedback data, thereby improving the accuracy of ad targeting.

[1200] Specific examples

[1201] For example, when a user is in a shopping mall, a smartphone app can analyze the user's emotional state through the camera. If the app determines that the user is having fun, it will deliver coupon advertisements for nearby cafes. If the user is tired, it will deliver advertisements for relaxation services.

[1202] Example prompt sentence:

[1203] "What kind of coupon ads can be delivered to users at the shopping mall to get the most impact when they are having fun?"

[1204] "When a user is tired in a shopping mall, what kind of relaxation service advertisement should be delivered to them to get the most effect?"

[1205] In this way, optimal advertisement delivery based on the user's emotional state is possible.

[1206] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1207] Step 1:

[1208] The server receives location information, app usage history, and emotional data obtained from the camera in real time from the smartphone. This data is converted into a standardized format (e.g., CSV file) and stored in a database.

[1209] Input: Location information from smartphone, app usage history, emotional data

[1210] Output: Data saved in a unified format (CSV file)

[1211] Step 2:

[1212] If there are missing parts in the imported data, the server will fill in the missing data by, for example, applying a forward fill method using the previous data value or a mean value interpolation method.

[1213] Input: Imported data, including missing data

[1214] Output: A complete and consistent dataset

[1215] Step 3:

[1216] The server analyzes the enriched dataset and uses a generative AI model to discover causal structures, a process that employs machine learning algorithms and statistical methods.

[1217] Input: Imputed dataset

[1218] Output: A structural model of the discovered causal relationships

[1219] Step 4:

[1220] The server constructs a Bayesian network based on a structural model of the discovered causal relationships and learns the conditional probabilities between each piece of data.

[1221] Input: structural model of causality

[1222] Output: Constructed Bayesian network and learned conditional probabilities

[1223] Step 5:

[1224] The server analyzes specific behavioral patterns and emotional states of users and generates optimal advertisements using a Bayesian network. It selects appropriate targeted advertisements based on the latest user data obtained from smartphones.

[1225] Input: User behavior patterns, emotional state, and Bayesian network

[1226] Output: The best ad generated

[1227] Step 6:

[1228] The server then delivers the generated advertisements to smartphones, adjusting the timing to ensure they are delivered at the appropriate time based on the user's current location and behavior.

[1229] Input: Generated ad, user's current location

[1230] Output: Advertisement delivered to the user's smartphone

[1231] Step 7:

[1232] The server evaluates the effectiveness of the delivered advertisements in real time and analyzes the collected feedback data. It collects user response data (click-through rate, duration of visit, etc.) and optimizes the advertisement content based on that data.

[1233] Input: Feedback data on advertising effectiveness

[1234] Output: Optimized advertising content and advertising strategies

[1235] This enables highly accurate ad delivery based on the user's behavioral patterns and emotional state, maximizing advertising effectiveness.

[1236] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1237] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1238] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1239] [Fourth embodiment]

[1240] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1241] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1242] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1243] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1244] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1245] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1246] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1247] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1248] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1249] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1250] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1251] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1252] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1253] The present invention provides a system for effectively utilizing big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[1254] 1. Data Acquisition and Preprocessing

[1255] The server captures IoT device data

[1256] The server ingests big data obtained from various IoT devices. This data usually comes in various formats and structures, so the server converts it into a unified format (e.g., CSV file) and ingests it into a DataFrame that can be analyzed.

[1257] The server completes the missing data

[1258] Since the ingested data is likely to contain missing values, the server performs preprocessing and imputes the missing values ​​in an appropriate way, for example, by using forward fill to maintain data consistency.

[1259] 2. Discovering causal structures

[1260] The server discovers the causal structure

[1261] After data preprocessing is complete, the server uses a generative AI model to discover the causal structure within the data. At this stage, advanced machine learning algorithms and statistical methods are used to uncover hidden causal relationships between data, and the resulting structural model is then used to build a Bayesian network.

[1262] 3. Bayesian Network Training

[1263] The server builds a Bayesian network and learns probability relationships.

[1264] The server builds a Bayesian network based on a structural model of causal relationships. The Bayesian network models the conditional probabilities between variables and establishes which variables influence which others. Through this learning process, the probabilistic relationships between each node (variable) are defined.

[1265] 4. Causal Analysis

[1266] The user specifies a specific event

[1267] When a user wants to analyze the causal relationship of a specific anomaly or event (e.g., temperature abnormality) in a piece of equipment, the user sends a query from the terminal to the server. This query includes details of the specific event they want to investigate.

[1268] The server analyzes the causal relationships

[1269] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. Specifically, it calculates the influence of other related nodes (variables) to identify the cause of the anomaly.

[1270] 5. Return of results and decision making

[1271] The server returns the analysis results to the device

[1272] Once the causal relationship analysis is complete, the server sends the results back to the device, allowing the user to create an appropriate maintenance plan or take appropriate measures to deal with problems.

[1273] Specific examples

[1274] For example, if a "temperature abnormality" occurs in a facility in a factory, the user uses the present invention to identify the cause. In this case, the following procedure is followed.

[1275] 1. The server collects IoT device data: The server collects sensor data (temperature, pressure, current, etc.) obtained from each piece of factory equipment and converts it into an analyzable format.

[1276] 2. The server completes missing data values: The server completes missing parts of the imported data to maintain data consistency.

[1277] 3. The server discovers the causal structure: The server uses a generative AI model to learn the causal relationships between the data and identify potential factors related to temperature anomalies.

[1278] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns a conditional probability table.

[1279] 5. The user specifies a specific event: The user sends an event called "temperature abnormality" from the terminal to the server as a query.

[1280] 6. Server analyzes causality: The server receives the query and analyzes the cause of the temperature anomaly using a Bayesian network.

[1281] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can then use the results to create an appropriate maintenance plan.

[1282] In this way, the present invention utilizes data held by a company to achieve efficient equipment management and troubleshooting.

[1283] The processing flow will be explained below.

[1284] Step 1:

[1285] The server imports IoT device data. Specifically, the server receives data obtained from various sensors and devices in a unified format such as CSV and converts it into an analyzable DataFrame. This data import is performed periodically, and data updates can also be made in real time.

[1286] Step 2:

[1287] The server completes missing data values. Because the imported data often contains missing values, the server performs preprocessing and completes the missing data in an appropriate manner. Specifically, it maintains data consistency by using techniques such as forward filling using previous data values.

[1288] Step 3:

[1289] The server discovers the causal structure. After preprocessing is complete, the server uses a generative AI model to explore causal relationships between the data. This involves using machine learning algorithms and statistical methods to generate a structural model that includes hidden causal relationships.

[1290] Step 4:

[1291] The server builds a Bayesian network and learns the probabilistic relationships. Based on the structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. In this learning process, the probabilistic relationships between each node are defined in detail.

[1292] Step 5:

[1293] The user specifies a specific event. The user enters a query about a specific event of interest (e.g., temperature anomaly) from the terminal and sends it to the server. This query contains details of the event they want to investigate.

[1294] Step 6:

[1295] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies other factors that may affect the event and evaluates the degree of influence.

[1296] Step 7:

[1297] The server returns the analysis results to the terminal. Once the causal relationship analysis results are obtained, the server sends the results back to the terminal. Based on these results, the user can create equipment maintenance plans and troubleshoot problems.

[1298] The above are the specific processing steps of the system of the present invention, which enable companies to effectively utilize big data and optimize their asset management through advanced causal analysis.

[1299] Example 1

[1300] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1301] Conventional systems make it difficult to efficiently utilize a company's big data and effectively carry out equipment maintenance and troubleshooting. In particular, converting data into a unified format, filling in missing data, and automatically discovering and analyzing causal relationships are often done manually, which is time-consuming and costly. It is also difficult to visualize the structure of causal relationships and quickly identify the factors that affect specific events. There is a need for a system that can solve this issue and improve the efficiency of equipment management and troubleshooting.

[1302] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1303] In this invention, the server includes means for importing big data and supplementing missing data, means for converting the imported data into a unified format and processing it into an analyzable structure, means for discovering a causal structure from preprocessed data using a generative AI model, means for building a Bayesian network based on the learned causal structure and learning probability relationships, and means for analyzing factors that influence specific events. This enables unified handling of big data in various formats held by a company, efficient data analysis, and rapid decision-making for equipment maintenance and troubleshooting.

[1304] "Big data" refers to large amounts of diverse, rapidly generated data sets that are difficult to process using traditional database management tools.

[1305] "Missing data" refers to values ​​that are missing from a dataset for some reason, and is often expressed in the form of NaN (Not a Number) or similar.

[1306] "Uniform format" refers to converting data from various formats into a consistent format (e.g., CSV, DataFrame, etc.), which makes data analysis easier.

[1307] "Analyzable structure" means that the data has a format and structure suitable for analysis, particularly a data frame or table format.

[1308] "Generative AI model" refers to a learning model generated using artificial intelligence, specifically including models built using machine learning or deep learning algorithms.

[1309] A "causal structure" refers to a model that shows the cause-and-effect relationship between data, visualizing how each element affects other elements.

[1310] A "Bayesian network" is a type of probabilistic graphical model that refers to a network used to represent conditional dependencies between variables.

[1311] A "probability relationship" refers to the conditional probability relationship between variables, and is a mathematical expression of the degree to which each variable affects other variables.

[1312] A "specific event" refers to a specific abnormality or phenomenon of interest to the user (e.g., temperature abnormality, machine failure, etc.), and causal analysis is performed based on this.

[1313] "Influencing factors" refer to elements or variables that have an effect on a particular event or outcome, and include multiple factors that may be causative.

[1314] "Analytical means" refers to the methods and processes used to perform data analysis, particularly using statistical methods and machine learning algorithms.

[1315] The present invention relates to a system that effectively utilizes big data held by a company to improve the efficiency of equipment maintenance and troubleshooting. Specific embodiments of the present invention will be described below.

[1316] Data acquisition and preprocessing

[1317] The server captures IoT device data

[1318] The server collects data in real time from various IoT devices. This data includes sensor data such as temperature, pressure, and current. Specifically, the server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. This operation allows the data to be managed in a consistent format, making subsequent analysis easier.

[1319] The server completes missing data

[1320] Since the imported data often contains missing values, the server uses the fillna method from the Pandas library to fill in the missing values. Specifically, it applies "dataframe.fillna(method='ffill')" to the DataFrame to fill in the missing values ​​with the previous value.

[1321] Discovery of causal structures

[1322] The server discovers the causal structure

[1323] After preprocessing is complete, the server uses a generative AI model to discover causal relationships. This process utilizes machine learning libraries such as Scikit-learn and TensorFlow to learn causal relationships based on the features of the data. Specifically, it uses Linear Regression models and Granger Causality Tests to identify causal relationships between data.

[1324] Bayesian Network Training

[1325] The server builds a Bayesian network and learns probability relationships.

[1326] Based on the discovered causal structure, the server uses libraries such as PyMC3 and pgmpy to build a Bayesian network, which derives the conditional probability between each variable (node). Specifically, use "import pymc3 as pm" to define the model and calculate the conditional probability.

[1327] Causal analysis

[1328] The user specifies a specific event

[1329] A user sends a query about a specific anomaly or event (e.g., temperature anomaly) from their device to the server. This query includes details of the event they want to investigate. For example, a user might enter "Please analyze the cause of the temperature anomaly from the sensor data from Factory A" through a web or mobile application and submit it.

[1330] The server analyzes the causal relationships

[1331] Based on the query received from the user, the server analyzes the cause of the event using the pre-built Bayesian network. It calculates the influence from other related nodes and identifies the cause of the anomaly. Specifically, it applies "pm.sample_posterior_predictive(trace, model=model)" to calculate the probability distribution of the anomaly factors.

[1332] Returning results and making decisions

[1333] The server returns the analysis results to the device

[1334] Once the analysis is complete, the server returns the results to the user's device as a JSON-formatted HTTP response. Based on these analysis results, the user can consider appropriate maintenance plans and troubleshooting measures.

[1335] This invention enables the unified handling of big data in various formats held by companies and efficient data analysis, thereby enabling rapid and accurate decision-making regarding equipment maintenance and troubleshooting.

[1336] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1337] Step 1:

[1338] Data Ingestion

[1339] The server collects data in real time from various IoT devices. As input, it uses information from each IoT device's API endpoint and message queue (e.g., MQTT). The server uses the Python "requests" library to obtain JSON-formatted data from each device's API, converts it to a CSV file, and saves it. The output is a CSV file in a unified format.

[1340] Specific behavior:

[1341] Get data from the IoT device using requests.get(api_endpoint).

[1342] Convert to CSV using data.to_csv('path / to / file.csv') and save.

[1343] Step 2:

[1344] Data Preprocessing

[1345] The server imputes missing values ​​in the imported data. The input is the CSV file obtained in step 1. The server uses the fillna method of the Pandas library to fill missing values ​​with the previous value. The output is a data frame with imputed missing values.

[1346] Specific behavior:

[1347] Read the CSV file with dataframe = pd.read_csv('path / to / file.csv').

[1348] Fill missing values ​​with dataframe.fillna(method='ffill').

[1349] Step 3:

[1350] Discovery of causal structures

[1351] Based on the preprocessed data, the server uses a generative AI model to discover the causal structure. The input is the data frame obtained in step 2. Causal relationships are learned using libraries such as Scikit-learn and TensorFlow. The output is a learned model that shows the causal relationships.

[1352] Specific behavior:

[1353] Define the model with model = LinearRegression().

[1354] Learn causal relationships with model.fit(X, y).

[1355] Step 4:

[1356] Bayesian Network Training

[1357] The server builds a Bayesian network based on the structural model of causal relationships and learns conditional probabilities. The input is the causal relationship model created in step 3. The Bayesian network is built using the PyMC3 and pgmpy libraries. The output is a Bayesian network probabilistic model.

[1358] Specific behavior:

[1359] Define the Bayesian network with pm.Model() as model:

[1360] Train a probabilistic model with pm.sample().

[1361] Step 5:

[1362] Receiving specific event analysis queries

[1363] The user sends a query about a specific anomaly or event (e.g., temperature anomaly) from a terminal to the server in the form of a prompt sentence. The input is the query text entered by the user. The output is the query sent to the server.

[1364] Specific behavior:

[1365] The user enters the following into the form on the terminal: "Please analyze the cause of the temperature abnormality from the sensor data of Factory A."

[1366] Click the submit button to send the query to the server.

[1367] Step 6:

[1368] Causal analysis

[1369] Based on the received query, the server analyzes the cause of the specified event using a Bayesian network. The inputs are the Bayesian network probabilistic model obtained in step 4 and the query received in step 5. The output is the analysis result of the anomaly cause.

[1370] Specific behavior:

[1371] Perform inference using pm.sample_posterior_predictive(trace, model=model).

[1372] Based on the query, determine the cause of the event.

[1373] Step 7:

[1374] Return of analysis results

[1375] The server returns the analysis results to the user's device. The input is the analysis results obtained in step 6. The output is the JSON data of the analysis results sent to the user.

[1376] Specific behavior:

[1377] Convert the parsed results into JSON format.

[1378] Return the response to the user device using response = HttpResponse(json.dumps(analysis result), content_type='application / json').

[1379] summary

[1380] In this way, the system automates the entire process from data collection to the return of analysis results, enabling efficient equipment management and troubleshooting.

[1381] (Application example 1)

[1382] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1383] There is a need to utilize big data in factory equipment to improve the efficiency of equipment maintenance and troubleshooting. However, currently, it is difficult to process and analyze huge amounts of data, and it is difficult to analyze missing data and causal relationships. Therefore, a system that can process data more efficiently and accurately and analyze anomalies is needed.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1385] In this invention, the server includes a means for importing big data and supplementing missing data, a means for discovering a causal structure from the imported data using a learning model, a means for constructing a Bayesian network based on the learned model and learning probability relationships, a means for analyzing factors that influence specific events, a means for collecting sensor data from robots in a factory, and a means for identifying abnormalities in the robots based on the collected data and recommending a maintenance plan, thereby enabling efficient and accurate data processing and abnormality analysis.

[1386] "Big data" refers to the collection of huge amounts of data that are difficult to process and analyze using conventional methods.

[1387] "Missing data" is data that indicates a lack of some of the information available in a dataset.

[1388] A "learning model" is an algorithm that learns patterns and relationships from data and makes predictions and classifications.

[1389] "Causality" is a concept that indicates that one event has a cause-and-effect relationship with another event.

[1390] A "Bayesian network" is a model that expresses probabilistic causal relationships using a graph structure, and is a method for expressing relationships between variables based on conditional probability.

[1391] An "IoT device" is a physical device connected to the Internet, equipped with sensors and actuators and capable of collecting and transmitting data.

[1392] A "factory robot" is a machine designed to perform automated tasks in a factory environment.

[1393] "Sensor data" is data that expresses physical phenomena and environmental conditions measured by a sensor as numerical values.

[1394] A "generative AI model" is an AI model that uses machine learning algorithms to perform generative tasks (e.g., text generation or image generation).

[1395] A "maintenance plan" is a plan for regular inspections and repairs to keep facilities and equipment in good condition.

[1396] "Identifying anomalies" means detecting behavior or conditions that deviate from normal operating conditions and clarifying the cause.

[1397] The present invention provides a system for implementing efficient maintenance and troubleshooting by utilizing big data obtained from robots and equipment in a factory. Specific embodiments of the present invention will be described below.

[1398] 1. Data Collection and Preprocessing

[1399] Data collection

[1400] The server collects sensor data from IoT devices in the factory, including data on the equipment's status, such as temperature, pressure, current, and vibration, and transmits the data to the server in real time over the network.

[1401] Data Preprocessing

[1402] Since the collected data may contain missing values, the server preprocesses the data. Specifically, it uses the pandas library to convert the data into DataFrame format and imputes missing values. In this process, it uses forward fill, which fills in the previous value.

[1403] 2. Discovering causal structures

[1404] Discovering causal relationships using generative AI models

[1405] After the preprocessing is complete, the server uses a generative AI model to discover the causal relationships between the data. An example of a generative AI model is OpenAI's GPT model. A prompt is sent to this model to analyze the causal relationships.

[1406] Example prompt sentence:

[1407] "Please find the causal relationship of the abnormal occurrence of the factory robot based on the following data:

[1408] data:

[1409] {

[1410] "timestamp": ["2023-10-01 12:00:00", "2023-10-01 12:01:00", "2023-10-01 12:02:00"],

[1411] "temperature": [45, 47, 48],

[1412] "pressure": [101, 102, 100],

[1413] "current": [5.5, 5.6, 5.3]

[1414] }

[1415] "

[1416] 3. Building a Bayesian network

[1417] Building Bayesian networks based on structure discovery

[1418] Based on the generated causal relationships, the server constructs a Bayesian network, which models the probabilistic relationships between variables and describes the impact of certain events on others.

[1419] 4. Analyze abnormalities and recommend maintenance plans

[1420] Anomaly analysis

[1421] When a user specifies a specific abnormal event (e.g., abnormal temperature) on their smartphone, the server uses a Bayesian network to analyze the cause of the abnormality and identify the main factors.

[1422] Notification of results and recommended maintenance plans

[1423] The analysis results are sent to the user's smartphone, allowing them to plan and carry out maintenance in a timely manner.

[1424] Hardware and software used

[1425] Hardware

[1426] IoT devices (sensors, actuators, etc.)

[1427] Server (high performance computer)

[1428] Smartphone device

[1429] software

[1430] pandas library (data preprocessing)

[1431] OpenAI's generative AI model (discovering causal structures)

[1432] Bayesian network construction software

[1433] By implementing this invention, it is possible to efficiently process and analyze the huge amount of data obtained from robots and equipment in a factory, identify the cause of abnormalities, and create an appropriate maintenance plan, thereby reducing the risk of equipment failure and improving operational efficiency.

[1434] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1435] Step 1: Collect data

[1436] The server collects sensor data from IoT devices in the factory (for example, robots and their peripherals). Input data includes temperature, pressure, current, vibration, etc., and this data is sent to the server in real time via the network. Specifically, the server periodically requests data from each sensor and aggregates it.

[1437] Step 2: Preprocessing the data

[1438] The server converts the received raw data into DataFrame format using the pandas library. Since the data may contain missing values, it imputes the missing values. Specifically, the server applies forward fill to the missing parts of the dataframe to create a consistent dataset. The output is clean data with missing values ​​imputed.

[1439] Step 3: Discover the causal structure

[1440] The server uses the preprocessed data to send a prompt to the generative AI model (e.g., GPT model). The prompt contains the JSON formatted content of the sensor data. The input is the clean data and the prompt, which the server sends to the generative AI model. The output is structural information about the causal relationships.

[1441] Step 4: Building a Bayesian network

[1442] The server constructs a Bayesian network based on the structural information of causal relationships obtained from the generative AI model. Specifically, the server calculates the conditional probability between each variable and expresses this in a graph structure. The input is the structural information of causal relationships, and the output is the constructed Bayesian network.

[1443] Step 5: Analyze the anomaly

[1444] The user specifies a specific abnormal event (e.g., abnormal temperature) from their smartphone device and sends a query to the server. The server then uses a Bayesian network to analyze the input abnormal event. Specifically, the server analyzes the related nodes in the Bayesian network and identifies the cause of the abnormality. The input is the query from the user, and the output is the result of identifying the cause of the abnormality.

[1445] Step 6: Communicate results and recommend a maintenance plan

[1446] The server notifies the user's smartphone of the analysis results it has obtained. The notification includes the cause of the anomaly and specific maintenance instructions accordingly. Specifically, the server formats the analysis results as a text message and sends it to the user's device. The input is the anomaly analysis result, and the output is a notification message to the user.

[1447] This will enable real-time monitoring of robots and equipment within the factory, early detection of abnormalities, and appropriate maintenance measures.

[1448] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1449] The present invention is a system that combines a system for collecting big data, completing missing data, and analyzing causal relationships with an emotion engine that recognizes user emotions. Specific embodiments of the present invention will be described below.

[1450] 1. Data Acquisition and Preprocessing

[1451] The server collects IoT device data and emotion data.

[1452] The server takes in data from various IoT devices and user emotion data obtained through the emotion engine. This data is converted into a unified format (e.g., CSV file) and saved as a DataFrame that can be analyzed.

[1453] The server completes the missing data

[1454] If there are gaps in the imported data, the server will fill them in using an appropriate method, such as forward filling using the previous data value, to maintain data consistency.

[1455] 2. Discovering causal structures

[1456] The server discovers the causal structure

[1457] The server then uses generative AI models on the preprocessed data to discover causal structures, using machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[1458] 3. Building and training a Bayesian network

[1459] The server builds a Bayesian network and learns probability relationships.

[1460] The server builds a Bayesian network based on a structural model of causal relationships and learns the conditional probabilities between each piece of data. This learning process defines in detail the probabilistic relationships between each node.

[1461] 4. Causal Analysis

[1462] The user specifies a specific event

[1463] The user enters a query about a particular event of interest (e.g., a temperature anomaly or a particular emotional state) and sends it to the server. This query includes details of the event they want to investigate.

[1464] The server analyzes the causal relationships

[1465] Based on the query received, the server uses a Bayesian network to analyze the causal relationships of the specified events, specifically by identifying other variables that may affect a particular event and assessing the degree of their influence.

[1466] 5. Return of results and decision making

[1467] The server returns the analysis results to the device

[1468] Once the results of the causal analysis are obtained, the server sends the results back to the device, allowing the user to create appropriate maintenance plans and troubleshooting measures based on these results.

[1469] Specific examples

[1470] For example, suppose a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken:

[1471] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data obtained via the emotion engine, and converts the data into an analyzable format.

[1472] 2. The server imputes missing values ​​in the data: The server imputes missing values ​​in the dataset using previous values ​​to maintain data consistency.

[1473] 3. The server discovers causal structures: The server uses a generative AI model to analyze the causal relationships in the data and identify the causal relationships related to temperature anomalies and employees' emotional states (stress levels).

[1474] 4. The server constructs a Bayesian network and learns the probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between the data.

[1475] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[1476] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[1477] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[1478] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[1479] The processing flow will be explained below.

[1480] Step 1:

[1481] The server takes in IoT device data and user emotion data. Specifically, the server receives sensor data from factories and emotion data from wearable devices worn by employees. It converts this data into a unified format (e.g., CSV file) and saves it in an analyzable DataFrame.

[1482] Step 2:

[1483] The server completes missing data. Because the imported data is likely to contain missing values, the server completes the missing values ​​in an appropriate manner. Specifically, the server maintains data consistency by using methods such as forward filling with data values ​​from previous points in time.

[1484] Step 3:

[1485] The server discovers the causal structure. Using the preprocessed data as input, the server uses a generative AI model to discover the causal structure between each piece of data. This process uses machine learning algorithms and statistical methods to identify hidden causal relationships between data.

[1486] Step 4:

[1487] The server builds a Bayesian network and learns the probabilistic relationships. Based on a structural model of causal relationships, the server models the conditional probability between each piece of data and builds a Bayesian network. This learning process defines in detail the probabilistic relationships between each node (variable).

[1488] Step 5:

[1489] The user specifies a specific event. The user inputs a specific event of interest (e.g., a query about a temperature anomaly or a specific emotional state) from their device and sends it to the server. This query contains details of the event they want to investigate.

[1490] Step 6:

[1491] The server analyzes the causal relationships of specific events. Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified event. Specifically, it identifies the influence of other variables related to the event and evaluates the degree of influence.

[1492] Step 7:

[1493] The server returns the analysis results to the device. Once the results of the causal relationship analysis are obtained, the server sends the results back to the device. Based on the analysis results, the user can create appropriate maintenance plans and troubleshooting measures.

[1494] Programmatic example

[1495] For example, if a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees, the user can do so as follows.

[1496] 1. The server takes in IoT device data and emotion data: The server takes in sensor data (e.g., temperature, humidity, current, etc.) obtained from each piece of factory equipment and employee emotion data (e.g., stress level, heart rate, etc.) obtained via the emotion engine, and converts the data into an analyzable format.

[1497] 2. The server completes missing data: The server completes missing values ​​in the imported data with data values ​​from the previous point in time to maintain consistency.

[1498] 3. The server discovers causal structures: The server uses a generative AI model to analyze causal relationships between data and identify causal relationships related to temperature anomalies and employee stress levels.

[1499] 4. The server constructs a Bayesian network and learns probability relationships: The server constructs a Bayesian network based on the causal structure model and learns the conditional probabilities between data.

[1500] 5. User specifies specific events: The user sends a query to the server from their terminal to investigate the events "temperature abnormality" and "employee stress level."

[1501] 6. The server analyzes the causal relationship: The server receives the query and uses a Bayesian network to analyze the causal relationship between temperature abnormalities and employee stress levels.

[1502] 7. The server returns the analysis results to the terminal: The analysis results are sent back to the user, who can use this information to develop appropriate maintenance measures and employee care plans.

[1503] In this way, the present invention provides a system that integrates big data and user emotional data, achieving both efficient facility management and employee emotional care.

[1504] Example 2

[1505] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1506] Conventional big data analysis systems not only require a huge amount of time and effort to import and preprocess data and discover causal relationships, but also insufficiently utilize emotional data. Furthermore, the process for users to analyze causal relationships based on specific events and gain actionable insights is ineffective. This makes it difficult to make appropriate decisions and efficiently manage facilities.

[1507] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for importing big data and complementing missing data, means for discovering a causal structure from the imported data using a generative AI model, means for constructing a Bayesian network based on the learned model and learning probabilistic relationships, means for a user to input a query regarding a specific event and for the server to analyze the causal relationships of the event, means for returning the analysis results to the user's terminal, and means for importing and analyzing emotion data. This enables a user to efficiently integrate big data and emotion data, analyze causal relationships based on specific events, and obtain practical insights in real time.

[1508] "Big data" refers to a set of data that is so large and complex that it cannot be processed using conventional data processing technology.

[1509] "Missing data" refers to data in a dataset where some values ​​are missing or incomplete.

[1510] A "generative AI model" refers to an artificial intelligence model that is used to learn from data and generate new predictions and insights.

[1511] A "Bayesian network" refers to a graphical model for representing causal relationships between random variables and making inferences using conditional probabilities.

[1512] "Emotional data" refers to data that represents the user's emotional state (e.g., stress, satisfaction, fatigue, etc.).

[1513] A "query" is a question or request made to a database or system to retrieve or manipulate information.

[1514] "Causality" refers to the direct effect or relationship that a change in one variable has on another variable.

[1515] "Analysis" refers to the process of examining data in detail and uncovering hidden meanings and relationships within it.

[1516] "User" refers to a person or organization that uses the system.

[1517] "Terminal" refers to a device (e.g., PC, tablet, smartphone, etc.) that a user uses to access and operate the system.

[1518] The present invention relates to a system that collects big data, integrates user emotion data, and analyzes causal relationships. This system incorporates big data, complements missing data, discovers causal relationship structures, builds Bayesian networks, analyzes causal relationships based on specific events, returns analysis results, and analyzes emotion data. Specific embodiments of the present invention are described in detail below.

[1519] Data acquisition and preprocessing

[1520] The server collects IoT device data and emotion data.

[1521] The server collects various data using the factory's sensors (e.g., temperature sensors, humidity sensors, current sensors, etc.) and emotion engines (e.g., cameras and emotion recognition software). This data is converted to, for example, CSV format and saved as a DataFrame using Python's Pandas library. Specifically, the data is read using pd.read_csv('sensor_data.csv') and converted into a unified format.

[1522] The server converts the data into a unified format and stores it.

[1523] The collected data is converted into a unified format (e.g., date and time, sensor values), for example by unifying all timestamps to UTC and scaling data in different units. This converted data is saved as a DataFrame for further processing.

[1524] The server completes the missing data

[1525] Missing values ​​in the dataset are imputed using the Pandas fillna method, e.g., dataframe.fillna(method='ffill', inplace=True) , which imputes missing values ​​with the previous value and ensures data consistency.

[1526] Discovery of causal structures

[1527] The server uses a generative AI model to discover the causal structure

[1528] Once the preprocessing is complete, a generative AI model (e.g., a causal impact model) is used to discover the causal structure of the data. Specifically, a causal analysis algorithm is run as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period) .

[1529] Building and training Bayesian networks

[1530] The server builds a Bayesian network

[1531] Use the pgmpy library to build a Bayesian network based on a structural model. Define the network as follows: model = BayesianNetwork(structure) .

[1532] The server learns the probability relationships

[1533] A Bayesian network uses conditional probabilities to learn the probabilistic relationships between nodes. Specifically, it is trained using a data frame, such as model.fit(dataframe) .

[1534] Causal analysis and return of results

[1535] A user enters a query about a specific event

[1536] A user sends a query about a specific event (e.g., temperature abnormality or a specific emotional state) to the server via a terminal. For example, a user enters a query to investigate "employee stress levels when the temperature exceeds 30 degrees."

[1537] The server analyzes the causal relationships and returns the analysis results.

[1538] Based on the received query, the server uses a Bayesian network to analyze the causal relationships of the specified events. The analysis results are sent back to the user's device. Specifically, the server uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) .

[1539] Emotional data capture and analysis

[1540] The server captures and analyzes the emotional data.

[1541] Emotion data obtained from the emotion engine is collected in real time using IP cameras and emotion recognition software, which analyzes employees' emotional states (e.g., stress, satisfaction, fatigue) from their facial expressions and voice data.

[1542] Specific examples

[1543] For example, consider a case where a user wants to analyze the relationship between "temperature abnormalities" in factory equipment and the resulting stress levels of employees. In this case, the following steps are taken: The server imports sensor data (e.g., temperature, humidity, current, etc.) from each piece of factory equipment and employee emotion data obtained via an emotion engine. Then, it preprocesses the data and appropriately imputes missing data. Next, it uses a generative AI model to analyze the causal relationships in the data, constructs a Bayesian network, and learns the probabilistic relationships. The user sends a query to the server from their device to investigate the events "temperature abnormalities" and "employee stress levels," and the server analyzes the causal relationships using a Bayesian network. Finally, the analysis results are sent back to the user's device, allowing appropriate maintenance measures and employee care plans to be developed.

[1544] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1545] Step 1: Ingest your data

[1546] The server collects real-time data from various factory sensors (temperature sensors, humidity sensors, current sensors, etc.) and the emotion engine (camera and emotion recognition software). The input is the raw data from each sensor and the data obtained by the emotion engine, which is converted to CSV format and saved. The data is read as a DataFrame using the Pandas library, e.g., pd.read_csv('sensor_data.csv'). The output is a unified format DataFrame.

[1547] Step 2: Preprocessing the data

[1548] The server converts the read data into a unified format. Specifically, it converts all timestamps to UTC time and unifies units as necessary. The input is various sensor data and emotion data in CSV format, and it scales the data and converts timestamps. The output is a preprocessed DataFrame.

[1549] Step 3: Imputing missing data

[1550] The server uses Pandas' fillna method to fill missing values ​​in the dataset. Specifically, dataframe.fillna(method='ffill', inplace=True) fills missing parts with the previous value. The input is a preprocessed DataFrame, and the output is a DataFrame with missing data filled in.

[1551] Step 4: Discover the causal structure

[1552] The server uses the data after preprocessing and missing data imputation to discover the causal structure using a generative AI model. Specifically, it generates a model as follows: causal_impact = CausalImpact(dataframe, pre_period, post_period). The input is a DataFrame with missing data imputed, and the output is a model showing the causal relationships.

[1553] Step 5: Building a Bayesian network

[1554] The server uses the pgmpy library to build a Bayesian network based on a causal model. Specifically, the network is defined as model = BayesianNetwork(structure) . The input is the causal model, and the output is the structure of the Bayesian network.

[1555] Step 6: Learning Probability Relationships

[1556] The server uses a Bayesian network to learn the conditional probabilities between each node. Specifically, training is performed using a data frame using model.fit(dataframe) . The input is a DataFrame with the Bayesian network structure and missing data imputed, and the output is a Bayesian network with learned conditional probabilities.

[1557] Step 7: Enter your query

[1558] A user uses a device to send a query about a specific event (e.g., temperature anomalies or a specific emotional state) to the server. The input is the user-specified event query, and the output is the data sent to the server for analysis of the query.

[1559] Step 8: Causal analysis

[1560] The server uses a Bayesian network to analyze the causal relationships of specified events based on the query submitted by the user. For example, it uses the pgmpy inference engine to perform the analysis as follows: inference.query(variables=['StressLevel'], evidence={'Temperature': 30}) . The input is the user's query and a Bayesian network trained on conditional probabilities, and the output is the analysis result.

[1561] Step 9: Returning the analysis results

[1562] The server returns the analysis results to the user's device. The user can use this information to formulate appropriate maintenance measures and employee care plans. The input is the analysis results, and the output is the analysis results returned to the user's device.

[1563] Specific prompt examples:

[1564] "Analyze the fluctuations in employees' stress levels when the temperature exceeds 30 degrees."

[1565] (Application example 2)

[1566] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1567] Conventional advertising systems were unable to generate optimal ads by comprehensively analyzing users' behavioral patterns and emotional states, resulting in limited advertising effectiveness. Furthermore, there were problems with reduced targeting accuracy when acquired data was incomplete or causal analysis was insufficient. Furthermore, the lack of a mechanism for evaluating advertising effectiveness in real time and dynamically feeding back the results limited the efficiency of the system.

[1568] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1569] In this invention, the server includes means for importing big data and supplementing missing data, means for discovering a causal structure from the imported data using a learning model, means for constructing a Bayesian network based on the learned model and learning probability relationships, means for collecting and analyzing user behavioral data and emotional data, means for generating and delivering optimal advertisements based on the user's behavioral patterns and emotional state using the Bayesian network, and means for evaluating the effectiveness of the delivered advertisements and feeding back the results to optimize the advertisement content. This enables highly accurate advertisement targeting and real-time evaluation based on the user's behavioral patterns and emotional state.

[1570] "Big data" refers to large, complex, and diverse data sets, as well as the technologies and methods for efficiently processing and analyzing them.

[1571] "Missing data" refers to a portion of a dataset where necessary information is missing, or the absence of a particular data point.

[1572] A "learning model" is a mathematical model that uses machine learning algorithms to learn patterns in data and make predictions or classifications.

[1573] A causal relationship is a relationship in which one event or variable influences another.

[1574] A "Bayesian network" is a type of probabilistic graphical model used to represent conditional dependencies between variables.

[1575] "User behavioral data" refers to data about a user's behavior when using a specific device or service, including, for example, location information and app usage history.

[1576] "Emotional data" refers to data that expresses a user's emotional state as numerical values ​​or categories, and is obtained by analyzing facial expressions, tone of voice, etc.

[1577] "Advertising" refers to information promoting a particular product or service, and refers to promotions or messages sent to users.

[1578] "Targeting" refers to the technology and methods of pinpointing and delivering the most appropriate advertisements and information to specific groups or individuals.

[1579] "Real-time evaluation" refers to the process of evaluating and analyzing specific events or data the moment they occur.

[1580] "Feedback" refers to information used to modify or optimize the next action or process based on the results obtained from an action or process that has already been performed.

[1581] MODE FOR CARRYING OUT THE INVENTION

[1582] The present invention provides a system that integrates and analyzes big data and user emotional data, and generates and delivers optimal advertisements based on the user's behavioral patterns and emotional state. This system includes the following means and processing steps.

[1583] System Configuration

[1584] Hardware and software configuration

[1585] 1. Hardware:

[1586] Smartphone: Collects user behavioral data (e.g., location information, app usage history) and emotional data through the camera.

[1587] Server: Preprocesses data, analyzes causal relationships, builds Bayesian networks, and generates and delivers advertisements.

[1588] 2. Software:

[1589] Data acquisition module: The data collected from the smartphone is acquired by the server.

[1590] Preprocessing module: Imputing missing data and data transformation.

[1591] Causal analysis module: Analyzes causal relationships between data using generative AI models.

[1592] Bayesian Network Module: Constructs and learns a Bayesian network based on the discovered causal relationships.

[1593] Advertisement generation and delivery module: Generates optimal advertisements based on the user's behavioral patterns and emotional state and delivers them to the smartphone.

[1594] Effectiveness evaluation module: Evaluates the effectiveness of delivered ads in real time and optimizes the ad content based on the results.

[1595] System Processing and Data Flow

[1596] 1. Data Acquisition and Preprocessing

[1597] The server collects location information, app usage history, and emotion data obtained from the camera in real time from the smartphone, converts it into a unified format (e.g., CSV file), and stores it as an analyzable DataFrame after completing a missing data imputation process.

[1598] 2. Causal analysis

[1599] The server uses the generative AI model on the data after data ingestion to identify causal relationships between the data. Specifically, it uses machine learning algorithms and statistical methods to discover hidden causal relationships.

[1600] 3. Building and training a Bayesian network

[1601] Based on the discovered causal structure, a Bayesian network is constructed to learn the conditional probabilities between each data point. This process defines the probabilistic relationships between variables in detail.

[1602] 4. Ad generation and delivery

[1603] If the server determines that the user is in a specific behavioral pattern or emotional state, it uses a Bayesian network to generate optimal advertisements and deliver them to the smartphone. For example, if the server determines that the user is having fun, it will deliver a coupon advertisement for a nearby cafe.

[1604] 5. Evaluation and feedback of advertising effectiveness

[1605] The effectiveness of delivered ads is evaluated in real time and ad content is optimized based on collected feedback data, thereby improving the accuracy of ad targeting.

[1606] Specific examples

[1607] For example, when a user is in a shopping mall, a smartphone app can analyze the user's emotional state through the camera. If the app determines that the user is having fun, it will deliver coupon advertisements for nearby cafes. If the user is tired, it will deliver advertisements for relaxation services.

[1608] Example prompt sentence:

[1609] "What kind of coupon ads can be delivered to users at the shopping mall to get the most impact when they are having fun?"

[1610] "When a user is tired in a shopping mall, what kind of relaxation service advertisement should be delivered to them to get the most effect?"

[1611] In this way, optimal advertisement delivery based on the user's emotional state is possible.

[1612] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1613] Step 1:

[1614] The server receives location information, app usage history, and emotional data obtained from the camera in real time from the smartphone. This data is converted into a standardized format (e.g., CSV file) and stored in a database.

[1615] Input: Location information from smartphone, app usage history, emotional data

[1616] Output: Data saved in a unified format (CSV file)

[1617] Step 2:

[1618] If there are missing parts in the imported data, the server will fill in the missing data by, for example, applying a forward fill method using the previous data value or a mean value interpolation method.

[1619] Input: Imported data, including missing data

[1620] Output: A complete and consistent dataset

[1621] Step 3:

[1622] The server analyzes the enriched dataset and uses a generative AI model to discover causal structures, a process that employs machine learning algorithms and statistical methods.

[1623] Input: Imputed dataset

[1624] Output: A structural model of the discovered causal relationships

[1625] Step 4:

[1626] The server constructs a Bayesian network based on a structural model of the discovered causal relationships and learns the conditional probabilities between each piece of data.

[1627] Input: structural model of causality

[1628] Output: Constructed Bayesian network and learned conditional probabilities

[1629] Step 5:

[1630] The server analyzes specific behavioral patterns and emotional states of users and generates optimal advertisements using a Bayesian network. It selects appropriate targeted advertisements based on the latest user data obtained from smartphones.

[1631] Input: User behavior patterns, emotional state, and Bayesian network

[1632] Output: The best ad generated

[1633] Step 6:

[1634] The server then delivers the generated advertisements to smartphones, adjusting the timing to ensure they are delivered at the appropriate time based on the user's current location and behavior.

[1635] Input: Generated ad, user's current location

[1636] Output: Advertisement delivered to the user's smartphone

[1637] Step 7:

[1638] The server evaluates the effectiveness of the delivered advertisements in real time and analyzes the collected feedback data. It collects user response data (click-through rate, duration of visit, etc.) and optimizes the advertisement content based on that data.

[1639] Input: Feedback data on advertising effectiveness

[1640] Output: Optimized advertising content and advertising strategies

[1641] This enables highly accurate ad delivery based on the user's behavioral patterns and emotional state, maximizing advertising effectiveness.

[1642] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1643] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1644] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1645] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1646] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1647] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1648] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1649] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1650] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1651] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1652] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1653] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1654] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1655] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1656] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1657] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1658] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1659] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1660] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1661] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1662] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1663] The following is further disclosed regarding the above embodiment.

[1664] (Claim 1)

[1665] A means of incorporating big data and filling in missing data,

[1666] A means of discovering the structure of causal relationships from the captured data using a learning model, and

[1667] A method for constructing a Bayesian network based on the learned model and learning probability relationships;

[1668] A system that includes a means of analyzing the factors that influence a particular event.

[1669] (Claim 2)

[1670] The system of claim 1, which acquires and imports big data from IoT devices and complements missing data.

[1671] (Claim 3)

[1672] The system of claim 1, utilizing a generative AI model to discover causal structures.

[1673] "Example 1"

[1674] (Claim 1)

[1675] A means of incorporating big data and filling in missing data,

[1676] A means of converting the captured data into a unified format and processing it into an analyzable structure;

[1677] A means of discovering causal structures from preprocessed data using a generative AI model;

[1678] A method for constructing a Bayesian network based on the learned causal structure and learning probability relationships;

[1679] A system that includes a means of analyzing the factors that influence a particular event.

[1680] (Claim 2)

[1681] The system of claim 1, which acquires and imports big data from devices connected via the Internet and completes missing data.

[1682] (Claim 3)

[1683] 10. The system of claim 1, utilizing machine learning to discover causal structures.

[1684] "Application Example 1"

[1685] (Claim 1)

[1686] A means of incorporating big data and filling in missing data,

[1687] A means of discovering the structure of causal relationships from the captured data using a learning model, and

[1688] A method for constructing a Bayesian network based on the learned model and learning probability relationships;

[1689] A means of analyzing the factors that influence a particular event;

[1690] means for collecting sensor data from robots in a factory;

[1691] The system includes a means to identify robot anomalies and recommend maintenance plans based on collected data.

[1692] (Claim 2)

[1693] The system of claim 1, which acquires and imports big data from IoT devices and complements missing data.

[1694] (Claim 3)

[1695] The system of claim 1, utilizing a generative AI model to discover causal structures.

[1696] "Example 2: Combining Emotion Engines"

[1697] (Claim 1)

[1698] A means of incorporating big data and filling in missing data,

[1699] A means of discovering causal relationships from the captured data using a generative AI model, and

[1700] A method for constructing a Bayesian network based on the learned model and learning probability relationships;

[1701] a means for a user to enter a query about a particular event and for the server to analyze the causality of the event;

[1702] means for returning the analysis results to the user's terminal;

[1703] A means for capturing and analyzing emotion data;

[1704] A system including:

[1705] (Claim 2)

[1706] The system of claim 1, which acquires and imports big data from IoT devices and complements missing data.

[1707] (Claim 3)

[1708] The system of claim 1, utilizing a generative AI model to discover causal structures.

[1709] "Application example 2 when combining emotion engines"

[1710] (Claim 1)

[1711] A means of incorporating big data and filling in missing data,

[1712] A means of discovering the structure of causal relationships from the captured data using a learning model, and

[1713] A method for constructing a Bayesian network based on the learned model and learning probability relationships;

[1714] A means of analyzing the factors that influence a particular event;

[1715] means for collecting and analyzing user behavioral and emotional data;

[1716] A means for generating and delivering optimal advertisements based on a user's behavioral patterns and emotional state using a Bayesian network;

[1717] A means of evaluating the effectiveness of delivered advertisements and optimizing the content of advertisements by feeding back the results.

[1718] A system including:

[1719] (Claim 2)

[1720] The system of claim 1, which acquires and imports big data from IoT devices and complements missing data.

[1721] (Claim 3)

[1722] The system of claim 1, utilizing a generative AI model to discover causal structures. [Explanation of symbols]

[1723] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of incorporating big data and filling in missing data, A means of discovering the structure of causal relationships from the captured data using a learning model, and A method for constructing a Bayesian network based on the learned model and learning probability relationships; A system that includes a means of analyzing the factors that influence a particular event.

2. The system according to claim 1, which acquires and imports big data from IoT devices and complements missing data.

3. The system of claim 1 , utilizing a generative AI model to discover causal structures.

Images