system

Abstract

We provide the system. [Solution] A means of acquiring information for real-time monitoring of the manufacturing process, An analysis means including a machine learning algorithm for analyzing the acquired information, A means of communication that detects anomalies based on analysis results and immediately notifies the administrator, A suggestion mechanism that recommends adjustments to the notified anomaly and provides a suggested operation, Implementation means for performing the adjusted work and monitoring it again, A user interface means for visually displaying anomaly detection information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Modern cell manufacturing processes rely on complex manual operations, resulting in variations in product quality and soaring labor costs. In particular, when there is a shortage of skilled technicians, manufacturing efficiency may decrease and competitiveness may be eroded. Such a situation hinders business expansion, and there is a need for a new system for efficiently mass-producing high-quality cell products.

Means for Solving the Problems

[0005] This invention provides a data acquisition means for monitoring the manufacturing process in real time, and improves the efficiency of the manufacturing process by using an analysis means including artificial intelligence to analyze the acquired data. It includes a notification means for detecting and notifying of abnormalities based on the analysis results, and a suggestion means for making suggestions in response. Furthermore, it provides a system that includes an execution means for putting the adjusted conditions into action and monitoring again, thereby automating the manufacturing process and enabling the production of high-quality and efficient cell products.

[0006] The "manufacturing process" refers to the series of steps involved in producing cell products, encompassing all procedures from raw materials to the final product.

[0007] "Real-time monitoring" refers to the immediate observation of the progress and status of the manufacturing process and the continuous acquisition of the latest data.

[0008] "Data acquisition means" refers to devices and systems for collecting various parameters in the manufacturing process, and includes sensors and monitoring devices.

[0009] "Artificial intelligence" refers to technologies that use machine learning and algorithms to automatically analyze data and make judgments about situations.

[0010] "Analysis means" refers to technologies and equipment used to analyze acquired data and evaluate the status of the manufacturing process.

[0011] "Anomaly detection" refers to finding deviations from standards in a manufacturing process and identifying factors that could potentially affect product quality or the process itself.

[0012] "Notification means" refers to devices or systems that transmit information to the user when an anomaly is detected.

[0013] "Proposed means" refers to techniques that provide specific action plans or adjustment plans to resolve detected anomalies.

[0014] "Implementation means" refers to devices or systems for applying the proposed adjustments to the manufacturing process and monitoring the results again. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention embodies a system for efficiently managing cell manufacturing processes and providing high-quality products. This system has the capability to monitor the manufacturing process in real time, detect anomalies, and propose appropriate countermeasures.

[0037] First, the terminal uses various sensors to acquire real-time data from the manufacturing line. This includes temperature, humidity, and cell growth rate, and this data is constantly transmitted to the server in an up-to-date state.

[0038] The server receives this data and immediately uses artificial intelligence to analyze it. The AI ​​employs learning algorithms to detect outliers and deviations from predictions. For example, it identifies anomalies when cell growth is slower than predicted or when environmental conditions fall outside the set range.

[0039] When an anomaly is detected, the server immediately transmits the information to the user through a notification system. For example, if the temperature exceeds the set range, the user receives a notification on their device and can review suggestions for adjusting the temperature.

[0040] The proposed adjustments are implemented in the manufacturing process in real time. Users can approve or adjust the optimal operations suggested by the AI ​​via a terminal. In this way, the entire manufacturing process is highly controlled, ensuring the production of uniform, high-quality products.

[0041] As a concrete example, let's assume a user is manufacturing a new batch of gene therapy drug. The terminal continuously generates and transmits manufacturing environment data, which is then analyzed by AI on the server. If there are no abnormalities, manufacturing proceeds without problems. However, if, for example, the pH becomes unstable, a notification is sent immediately, and adjustment suggestions are sent to the user. The user then makes the adjustments via the terminal, keeping the manufacturing process optimal. In this way, the entire system is automated, ensuring efficiency and product consistency.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The terminal acquires real-time data such as temperature, humidity, pH, and cell count from sensors installed on the production line. This data is recorded at regular intervals and prepared for the next analysis step.

[0045] Step 2:

[0046] The terminal sequentially transmits the acquired data to the server. The server quickly receives the large amount of data and stores it in a database in preparation for analysis.

[0047] Step 3:

[0048] The server executes artificial intelligence based on the received data and begins analyzing it. During this process, the AI ​​detects anomalies by comparing the current data with past data and evaluates the deviation between the set baseline values ​​and the measured values.

[0049] Step 4:

[0050] When the server detects an anomaly based on the analysis results, it immediately generates a notification message and sends it to the user. This notification includes the location of the anomaly, the estimated cause, and recommended countermeasures.

[0051] Step 5:

[0052] The user receives a notification from the server via their device and confirms the nature of the anomaly. Simultaneously, the user has the opportunity to consider the proposed solutions.

[0053] Step 6:

[0054] The user approves and applies the proposed adjustments on the terminal. Once the user implements the adjustments, the terminal applies the new settings to the production line and monitors the process again.

[0055] Step 7:

[0056] The server continues to monitor real-time data even after adjustments are made, evaluating whether the manufacturing process is proceeding optimally. This ensures that the entire system is continuously optimized.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] In manufacturing processes, where real-time information gathering, anomaly detection, and rapid response are required, conventional systems have challenges in overall efficiency and accuracy because each process—data acquisition, analysis, notification, and proposal—is managed individually. Furthermore, the inability to respond quickly when anomalies occur can compromise product quality and process efficiency.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes a device means for collecting environmental information using measuring devices placed in the manufacturing process and transmitting it via communication means; an analysis means having a device for preprocessing the received information and analyzing it using a machine learning model; and a perception means for presenting anomalies detected from the analysis results using an output device and prompting human intervention. This enables integrated management of the entire manufacturing process, real-time anomaly detection, and rapid response.

[0062] A "manufacturing process" is a set of sequential tasks or procedures designed to produce an item.

[0063] A "measuring device" is a device used to measure physical or chemical quantities, and specifically includes equipment for acquiring environmental information such as temperature and humidity.

[0064] "Communication means" refers to a technology or method for exchanging information between different devices, enabling the transmission and reception of data.

[0065] "Analysis means" refers to technologies that have a process for processing and understanding collected data, and typically involve using computer programs to analyze the data.

[0066] A "machine learning model" is a set of algorithms that have the ability to learn from data and recognize patterns, and is particularly used to detect anomalies.

[0067] An "output device" is a collection of hardware that displays processed data and information in a format that humans can understand.

[0068] "Perceptual means" refers to functions and devices that facilitate human judgment and actions based on analyzed information.

[0069] A "corrective procedure" is a set of specific operations and steps taken to improve a detected problem or anomaly.

[0070] "Feedback" is the process of incorporating the results of operations and procedures performed back into the system to help with future operations and improvements.

[0071] This invention is an integrated system aimed at efficient management of the manufacturing process and improvement of quality. The system consists of a server, terminals, and users. The specific implementation will be described below.

[0072] The terminal's role is to collect environmental information using measuring devices placed on the manufacturing line. These measuring devices include temperature sensors, humidity sensors, and optical sensors, which are used to acquire real-time data. This data is transmitted to a server via communication means. The hardware utilizes state-of-the-art communication modules and sensors.

[0073] The server functions as a hub for efficiently processing received data. First, it preprocesses the data, and then performs data analysis using machine learning models. Frameworks such as TENSORFLOW® and PyTorch are used for this analysis. Based on the analysis results, the server detects anomalies and presents this information to the user through perceptual means.

[0074] Users can make decisions based on the information presented and make adjustments according to the correction procedures suggested by the system. Users perform these adjustments via a terminal, optimizing various conditions in the manufacturing environment. The results are sent to the server as feedback and used to train the AI ​​model. This allows the server to improve the accuracy of anomaly detection in subsequent attempts.

[0075] As a concrete example, consider a scenario where a user is manufacturing a new drug. The terminal collects data such as temperature and pH level in real time and sends it to a server. The AI ​​on the server analyzes this data, and if it detects that the temperature exceeds the expected range, the user is immediately notified. The user then uses the terminal to operate the cooling system, referring to the suggested temperature adjustment plan, and returns the temperature to the specified range.

[0076] Examples of prompts to input into a generating AI model include, "Design a system that notifies when temperature data exceeds a set range," and "Describe an AI system that detects abnormal cell growth in real time and generates suggestions." These prompts help to show how the system operates under specific conditions.

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The terminal uses measuring devices installed on the manufacturing line to collect environmental information in real time. Specifically, temperature and humidity sensors operate and continuously measure data. The input data collected includes physical conditions such as temperature, humidity, and cell growth rate. This data is converted into digital signals and immediately transmitted to the server. The output is a packet of digital data received by the server.

[0080] Step 2:

[0081] The server receives digital data transmitted from the terminal. Using the received data as input, it first performs preprocessing such as standardizing the data format and removing noise. For example, it might standardize units for temperature data or filter out outliers. This outputs a clean dataset for analysis. This dataset forms the basis for analysis by the AI ​​model.

[0082] Step 3:

[0083] The server inputs preprocessed data into a machine learning model for analysis. This analysis utilizes anomaly detection models based on TensorFlow or PyTorch. The analysis specifically involves learning data patterns and detecting unexpected fluctuations and anomalies. The output is the anomaly detection result, which provides information to determine whether or not an anomaly exists.

[0084] Step 4:

[0085] The server notifies the user through sensory means based on the anomaly detection results. Specifically, it displays a warning on the console screen or mobile device, notifying the user of the location and nature of the anomaly. For example, the user might be informed that "the temperature has exceeded the set value." The input is the anomaly detection result, and the output is the specific notification information for the user.

[0086] Step 5:

[0087] The user considers necessary adjustments based on notifications received through the terminal. The user confirms the system's suggestion (e.g., lower the cooling device temperature setting by 2 degrees) and performs the action. Specifically, the user controls the actuator and changes the setting by pressing the adjustment button on the terminal screen. The inputs are notification information and suggestions, and the output is the adjusted manufacturing conditions.

[0088] Step 6:

[0089] The server receives the adjustment results made by the user as feedback and stores them in a database. During this process, conditions for re-execution and new insights are accumulated. Specifically, the AI ​​model is adjusted and trained based on past feedback, improving the accuracy of subsequent analyses. The input to this process is user feedback, and the output is updated feedback data.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] As automation of monitoring and management progresses in factory manufacturing processes, real-time anomaly detection and immediate response are required. However, current systems suffer from low accuracy in anomaly detection and delayed responses, leading to decreased production efficiency and increased product defect rates. Therefore, a system is needed that can detect anomalies in real time with high accuracy and enable rapid and optimal responses.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes information acquisition means for real-time monitoring of the manufacturing process, analysis means including a machine learning algorithm for analyzing the acquired information, and communication means for detecting anomalies based on the analysis results and immediately notifying the administrator. This enables advanced anomaly detection, rapid notification, and response in real time.

[0095] An "information acquisition means" is a device that has the function of collecting environmental information such as temperature and humidity in real time using sensors during the manufacturing process within a factory.

[0096] A "machine learning algorithm" is an artificial intelligence technology used to analyze acquired data and detect or predict anomalies.

[0097] A "communication method" refers to a means of communication used to immediately notify administrators of anomalies detected through analysis and prompt them to take appropriate action.

[0098] A "suggestion system" is a system that has the function of providing the worker with the most suitable countermeasures for the detected anomaly.

[0099] "Implementation means" refers to equipment that carries out the proposed adjustments and operations in the manufacturing process and manages the process to ensure it continues properly.

[0100] A "user interface" is an interactive screen that visually displays anomaly detection information, allowing administrators and workers to easily understand the situation and take appropriate action.

[0101] The system that realizes this invention is designed to efficiently manage manufacturing processes in a factory. The server is equipped with information acquisition means that acquire information from the manufacturing line in real time using various sensors. These include temperature sensors, humidity sensors, and pH sensors. The acquired information is immediately transmitted to the server and analyzed using a Python®-based machine learning algorithm. This analysis is performed using AI frameworks such as TensorFlow and PyTorch, and if an anomaly is detected, the administrator is notified by a communication means.

[0102] Users can visually confirm this anomaly through the provided user interface. This interface utilizes data visualization libraries such as Plotly to display the details of the anomaly and the proposed adjustments. Furthermore, the suggestion mechanism presents the optimal adjustment for the detected anomaly, allowing the user to manage the process based on it.

[0103] As a concrete example, consider a scenario where the temperature exceeds a set range during cell culture in a factory. When a sensor detects a mutation, the information is sent to a server, analyzed by AI, and then notified to the administrator's terminal. Simultaneously, specific adjustment suggestions are displayed, and the manufacturing environment is immediately optimized through user intervention.

[0104] Examples of prompt statements include commands that consider specific scenarios, such as, "Develop an AI model to adjust the production line if the temperature exceeds the appropriate range." In this way, advanced real-time production management and rapid response become possible.

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The server acquires data in real time from various sensors installed within the factory. The sensors measure environmental data such as temperature, humidity, and pH values, and transmit this data to the server. The input is the measurement data from the sensors, and the output is aggregated on the server.

[0108] Step 2:

[0109] The server analyzes the acquired data using machine learning algorithms. It utilizes AI frameworks such as TensorFlow and PyTorch to input data into an anomaly detection model. The AI ​​model identifies anomalies by comparing the data against set thresholds. The input is data from sensors, and the output is the analysis results.

[0110] Step 3:

[0111] When an anomaly is detected, the server immediately sends a notification to the administrator via a communication channel. This notification includes details of the detected anomaly and its impact. The input is the result of AI analysis, and the output is a notification to the user. Specifically, the operation involves generating and sending an alert message.

[0112] Step 4:

[0113] The user receives notification content on the user interface and visually confirms the anomaly. Visualization libraries such as Plotly are used to display the anomaly patterns and timing. The input is a notification from the server, and the output is visualized data. Specifically, this involves viewing the data on the display.

[0114] Step 5:

[0115] The user reviews the proposed adjustments presented by the suggested method and selects the necessary actions. The suggestions include specific procedures for the optimal operation or configuration changes in response to an anomaly. The input is the proposed adjustments, and the output is the user's selection. The specific actions involve reviewing the adjustments and selecting actions.

[0116] Step 6:

[0117] The server optimizes the manufacturing process by performing adjustments using implementation methods based on user selections. This ensures that the manufacturing environment is properly maintained and product quality is preserved. The input is the user's adjustment instructions, and the output is the result of the performed adjustments. Specifically, this involves executing changes to the settings of the manufacturing line.

[0118] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0119] This invention provides a system that recognizes the user's emotional state and reflects it in the management of the manufacturing process in order to improve manufacturing efficiency and quality in the cell manufacturing process. Specifically, in addition to basic data acquisition means, analysis means, notification means, suggestion means, and execution means for managing the manufacturing process, it integrates an emotion engine that analyzes the user's emotions.

[0120] First, the terminal acquires various data from the manufacturing line and sends it to the server. The server uses artificial intelligence to analyze the data and detect anomalies. If an anomaly is detected, the information is immediately communicated to the user through a notification system.

[0121] This is where the emotion engine comes in. Installed on the server, the emotion engine analyzes the user's behavior and reactions when using the interface and infers the user's emotional state. For example, it can detect stress and anxiety from factors such as operation speed, error frequency, and facial expressions using facial recognition technology.

[0122] Subsequently, the emotion engine operates, along with suggestion methods, that are appropriate according to the user's emotions. If the user's stress level is determined to be high, the tone of the notification can be softened and the explanation can be made more detailed to reduce the user's burden. This allows the user to perform the adjustment work with peace of mind.

[0123] As a concrete example, if a user makes an error during the cell manufacturing process, the emotion engine detects the user's stress level. When the server notifies the user of the anomaly, it sends a notification to the user that includes an encouraging message based on the emotion engine's analysis. In this way, the user can maintain composure and smoothly adjust the manufacturing process. This system enables integrated management of the cell manufacturing process while taking the user's psychological state into consideration.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] The terminal collects data such as temperature, humidity, pH, and cell count in real time from sensors placed throughout the production line. This data is used to monitor the production conditions.

[0127] Step 2:

[0128] The terminal sends the collected data to the server. The server receives this data, stores it internally, and prepares it for analysis.

[0129] Step 3:

[0130] The server uses artificial intelligence to analyze the data. The AI ​​executes an algorithm that detects anomalies by comparing the data with past data. Here, it identifies parameters that fall outside the normal range.

[0131] Step 4:

[0132] When the server detects an anomaly as a result of data analysis, it immediately notifies the user of the anomaly via a notification system. This notification includes the type of anomaly and recommended countermeasures.

[0133] Step 5:

[0134] The server's emotion engine analyzes the user's emotional state based on data such as the user's operation history and device camera. The AI ​​may detect the user's stress, anxiety, and other emotional states.

[0135] Step 6:

[0136] The server takes into account the user's emotional state, as assessed by the emotion engine, and adjusts the tone of notifications accordingly. For example, if the user is judged to be highly stressed, the notification message can be made concise and easy to understand, and may include an encouraging message.

[0137] Step 7:

[0138] Users check notifications sent from the server via their devices. These notifications include thoughtful instructions based on the analysis results of the emotion engine, allowing users to calmly choose the appropriate course of action.

[0139] Step 8:

[0140] The user implements the suggested adjustments on the terminal. The terminal applies the settings selected by the user to the production line and begins collecting new data.

[0141] Step 9:

[0142] The server will continue data analysis to confirm that the production line is operating normally in its adjusted state. This will help stabilize the manufacturing process.

[0143] (Example 2)

[0144] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0145] In manufacturing processes, real-time monitoring and immediate response to anomalies are crucial, but these are ineffective with conventional systems that do not adapt to the user's psychological state. In particular, appropriate responses are required in situations where users experience stress. Therefore, there is a need to provide a system that can respond based on the user's emotional state while maintaining the efficiency and quality of the manufacturing process.

[0146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0147] In this invention, the server includes an information acquisition device means for monitoring the manufacturing process in real time, an analysis means including machine learning for analyzing the acquired information, and a means for using an emotion analysis device for analyzing the user's emotional state. This enables abnormal response and optimization of the manufacturing process that are tailored to the user's emotional state.

[0148] An "information acquisition device" is a device that collects data in real time using various sensors and measuring instruments in the manufacturing process.

[0149] "Machine learning" is a field of artificial intelligence that involves analyzing data and recognizing trends and patterns within it.

[0150] "Analysis methods" refer to the processes and techniques used to analyze acquired data and identify anomalies and trends.

[0151] A "notification device" is a device that has the function of notifying the user when an abnormality is detected.

[0152] A "suggestion device" is a system component that presents the user with the optimal countermeasure for an anomaly.

[0153] An "implementation device" is a device that reflects the conditions adjusted based on the proposal into the manufacturing process and has the function of monitoring the results again.

[0154] An "emotion analysis device" is a device that estimates a user's emotional state based on their behavior and facial expressions.

[0155] This invention is an integrated system that optimizes efficiency and quality in the manufacturing process and enables responses based on the user's psychological state. The system comprises an information acquisition device, an analysis means including machine learning, a notification device, a suggestion device, an implementation device, and an emotion analysis device that analyzes the user's emotional state.

[0156] First, the terminal acquires data in real time via sensors and devices installed on the manufacturing line and transmits it to a server. This data includes temperature, humidity, pressure, and operating status.

[0157] The server analyzes data acquired using a generative AI model and applies machine learning techniques to detect anomalies. This allows for the rapid identification of anomalies, ensuring that the manufacturing process is maintained optimally.

[0158] When an anomaly is detected, the server immediately notifies the user through an alerting device. In addition, an emotion analysis device analyzes the user's emotional state in real time, and uses the analysis results to adapt and adjust the tone of the notification and response measures. This allows for a response that minimizes the burden on the user, even when they are feeling stressed or anxious.

[0159] For example, if the temperature shows an abnormal value during the manufacturing process, the notification will include a prompt to check the cooling system, along with a message such as, "Please remain calm. We will provide you with detailed instructions."

[0160] A concrete example of a prompt message would be: "A data anomaly has been detected on the production line. The user may be feeling anxious. What reassuring message should be sent?"

[0161] In this way, the server and terminal work together to create a system that enables the smooth operation of the manufacturing process while taking into account the user's emotional state.

[0162] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0163] Step 1:

[0164] The terminal collects data from various sensors placed on the manufacturing line. Specifically, temperature, humidity, pressure, and machine operating status are acquired in real time. This input data is collected as basic information for monitoring the manufacturing process.

[0165] Step 2:

[0166] The terminal sends the collected data to the server. The data is transmitted via a communication protocol and delivered to the server in real time over the network. Through this process, the server obtains the data for analysis.

[0167] Step 3:

[0168] The server analyzes the received data using a generating AI model. Specifically, machine learning techniques are applied to detect anomaly patterns in the data. It analyzes real-time information as input data, and if an anomaly is detected, it outputs the result as an anomaly summary.

[0169] Step 4:

[0170] The server immediately notifies the user via an alerting device if an anomaly is detected. The notification includes an explanation of the anomaly, specific countermeasures, and encouraging or cautionary messages generated by a generative AI model. In this step, the notification content is adjusted based on the results of sentiment analysis, aiming to reduce the user's psychological burden.

[0171] Step 5:

[0172] Upon receiving a notification, the user receives guidance from the sentiment analysis device and makes appropriate adjustments. Based on the feedback from the sentiment analysis device, the server provides the next optimal prompts and suggestions to support the user. The user's behavior and responses are continuously monitored, and the sentiment data is used for further analysis.

[0173] This series of steps creates a system where servers and terminals work together to provide users with optimal guidance for the manufacturing process.

[0174] (Application Example 2)

[0175] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0176] Traditional manufacturing process management lacks consideration for the emotional state of users, leading to problems such as decreased work efficiency and quality. Furthermore, the lack of information provided to enable appropriate responses when anomalies are detected can cause users to experience excessive stress.

[0177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0178] In this invention, the server includes means for acquiring information to perform real-time observation of the manufacturing process, means for analyzing the acquired information including machine learning, and means for integrating an emotion analysis engine to analyze the user's emotional state and provide optimal support according to the user's emotions. This enables efficient and high-quality management of the manufacturing process while taking the user's emotional state into consideration.

[0179] "Information acquisition means for real-time observation of the manufacturing process" refers to means for collecting necessary data in real time during manufacturing operations and processing it immediately.

[0180] "Analysis methods including machine learning to analyze acquired information" refers to methods that incorporate machine learning techniques to detect anomalies or patterns hidden within collected data.

[0181] "A means of integrating an emotion analysis engine that analyzes the user's emotional state and providing optimal support according to the user's emotions" refers to a means of evaluating the user's emotional state by analyzing their facial expressions and operation patterns, and then providing support accordingly.

[0182] The system for implementing this invention can acquire and analyze information during the manufacturing process and provide appropriate support tailored to the user's emotional state. The server uses information acquisition means to collect data from the manufacturing process in real time. This information is expected to be provided from terminals equipped with sensors and cameras. The acquired data is then analyzed by analysis means including machine learning. In this process, machine learning algorithms are used to detect known anomalies and patterns that can be improved.

[0183] Furthermore, an emotion analysis engine is integrated to analyze the user's emotional state. This engine analyzes behavioral data such as the user's facial expressions, operation speed, and error frequency, and uses this to infer the user's emotional state. The results of this analysis are used to provide the user with the best possible support. For example, if the user is feeling stressed, the notification tone may be softened and encouraging messages added to reduce the user's burden.

[0184] This system uses the OpenCV library and Keras to perform facial recognition and emotion analysis. Based on the analysis results, an SDK can be used for robot control to provide appropriate communication. For example, if a worker on a manufacturing line shows signs of fatigue, the emotion analysis engine can detect that emotion, and the robot can deliver a message such as, "You've worked hard. Why don't you take a short break?" An example of a prompt message to input into the AI ​​model generated based on this invention is, "Consider a factory robot system that analyzes an employee's facial expression to determine their emotion and provides optimal support if they are stressed."

[0185] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0186] Step 1:

[0187] The server receives data in real time from terminals equipped with sensors and cameras on the manufacturing line. This input data includes information about the progress of manufacturing and the facial expressions of workers. This data is first stored in a database and used as foundational data for subsequent analysis.

[0188] Step 2:

[0189] The server analyzes the acquired data using machine learning algorithms to detect signs of anomalies. This analysis includes data processing and classification to identify abnormal operating patterns and product quality degradation. The output generates detailed information about the detected anomalies.

[0190] Step 3:

[0191] The server uses an emotion analysis engine to analyze the user's emotional state. Input data includes the user's operation speed, error frequency, and facial expression data. Through data processing, this data is output as the user's stress and fatigue levels.

[0192] Step 4:

[0193] The server generates optimal support for the user based on anomaly notifications and analysis of emotional state. Specifically, it generates notifications in a gentle tone and sends them with encouraging messages. This process uses a generative AI model, which generates different messages depending on the prompt.

[0194] Step 5:

[0195] After the user receives a notification, the server monitors whether adjustments have been made in response to the notification and provides further support as needed. Throughout the entire process, the terminal and server exchange information in real time to optimize system operation.

[0196] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0197] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0198] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0199] [Second Embodiment]

[0200] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0201] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0202] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0203] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0204] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0205] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0206] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0207] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0208] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0209] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0210] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0211] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0212] This invention embodies a system for efficiently managing cell manufacturing processes and providing high-quality products. This system has the capability to monitor the manufacturing process in real time, detect anomalies, and propose appropriate countermeasures.

[0213] First, the terminal uses various sensors to acquire real-time data from the manufacturing line. This includes temperature, humidity, and cell growth rate, and this data is constantly transmitted to the server in an up-to-date state.

[0214] The server receives this data and immediately uses artificial intelligence to analyze it. The AI ​​employs learning algorithms to detect outliers and deviations from predictions. For example, it identifies anomalies when cell growth is slower than predicted or when environmental conditions fall outside the set range.

[0215] When an anomaly is detected, the server immediately transmits the information to the user through a notification system. For example, if the temperature exceeds the set range, the user receives a notification on their device and can review suggestions for adjusting the temperature.

[0216] The proposed adjustments are implemented in the manufacturing process in real time. Users can approve or adjust the optimal operations suggested by the AI ​​via a terminal. In this way, the entire manufacturing process is highly controlled, ensuring the production of uniform, high-quality products.

[0217] As a concrete example, let's assume a user is manufacturing a new batch of gene therapy drug. The terminal continuously generates and transmits manufacturing environment data, which is then analyzed by AI on the server. If there are no abnormalities, manufacturing proceeds without problems. However, if, for example, the pH becomes unstable, a notification is sent immediately, and adjustment suggestions are sent to the user. The user then makes the adjustments via the terminal, keeping the manufacturing process optimal. In this way, the entire system is automated, ensuring efficiency and product consistency.

[0218] The following describes the processing flow.

[0219] Step 1:

[0220] The terminal acquires real-time data such as temperature, humidity, pH, and cell count from sensors installed on the production line. This data is recorded at regular intervals and prepared for the next analysis step.

[0221] Step 2:

[0222] The terminal sequentially transmits the acquired data to the server. The server quickly receives the large amount of data and stores it in a database in preparation for analysis.

[0223] Step 3:

[0224] The server executes artificial intelligence based on the received data and begins analyzing it. During this process, the AI ​​detects anomalies by comparing the current data with past data and evaluates the deviation between the set baseline values ​​and the measured values.

[0225] Step 4:

[0226] When the server detects an anomaly based on the analysis results, it immediately generates a notification message and sends it to the user. This notification includes the location of the anomaly, the estimated cause, and recommended countermeasures.

[0227] Step 5:

[0228] The user receives a notification from the server via their device and confirms the nature of the anomaly. Simultaneously, the user has the opportunity to consider the proposed solutions.

[0229] Step 6:

[0230] The user approves and applies the proposed adjustments on the terminal. Once the user implements the adjustments, the terminal applies the new settings to the production line and monitors the process again.

[0231] Step 7:

[0232] The server continues to monitor real-time data even after adjustments are made, evaluating whether the manufacturing process is proceeding optimally. This ensures that the entire system is continuously optimized.

[0233] (Example 1)

[0234] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0235] In manufacturing processes, where real-time information gathering, anomaly detection, and rapid response are required, conventional systems have challenges in overall efficiency and accuracy because each process—data acquisition, analysis, notification, and proposal—is managed individually. Furthermore, the inability to respond quickly when anomalies occur can compromise product quality and process efficiency.

[0236] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0237] In this invention, the server includes a device means for collecting environmental information using measuring devices placed in the manufacturing process and transmitting it via communication means; an analysis means having a device for preprocessing the received information and analyzing it using a machine learning model; and a perception means for presenting anomalies detected from the analysis results using an output device and prompting human intervention. This enables integrated management of the entire manufacturing process, real-time anomaly detection, and rapid response.

[0238] A "manufacturing process" is a set of sequential tasks or procedures designed to produce an item.

[0239] A "measuring device" is a device used to measure physical or chemical quantities, and specifically includes equipment for acquiring environmental information such as temperature and humidity.

[0240] "Communication means" refers to a technology or method for exchanging information between different devices, enabling the transmission and reception of data.

[0241] "Analysis means" refers to technologies that have a process for processing and understanding collected data, and typically involve using computer programs to analyze the data.

[0242] A "machine learning model" is a set of algorithms that have the ability to learn from data and recognize patterns, and is particularly used to detect anomalies.

[0243] An "output device" is a collection of hardware that displays processed data and information in a format that humans can understand.

[0244] "Perceptual means" refers to functions and devices that facilitate human judgment and actions based on analyzed information.

[0245] A "corrective procedure" is a set of specific operations and steps taken to improve a detected problem or anomaly.

[0246] "Feedback" is the process of incorporating the results of operations and procedures performed back into the system to help with future operations and improvements.

[0247] This invention is an integrated system aimed at efficient management of the manufacturing process and improvement of quality. The system consists of a server, terminals, and users. The specific implementation will be described below.

[0248] The terminal's role is to collect environmental information using measuring devices placed on the manufacturing line. These measuring devices include temperature sensors, humidity sensors, and optical sensors, which are used to acquire real-time data. This data is transmitted to a server via communication means. The hardware utilizes state-of-the-art communication modules and sensors.

[0249] The server functions as a hub for efficiently processing received data. First, it preprocesses the data, and then performs data analysis using machine learning models. Frameworks such as TensorFlow and PyTorch are used for this analysis. Based on the analysis results, the server detects anomalies and presents this information to the user through perceptual means.

[0250] Users can make decisions based on the information presented and make adjustments according to the correction procedures suggested by the system. Users perform these adjustments via a terminal, optimizing various conditions in the manufacturing environment. The results are sent to the server as feedback and used to train the AI ​​model. This allows the server to improve the accuracy of anomaly detection in subsequent attempts.

[0251] As a concrete example, consider a scenario where a user is manufacturing a new drug. The terminal collects data such as temperature and pH level in real time and sends it to a server. The AI ​​on the server analyzes this data, and if it detects that the temperature exceeds the expected range, the user is immediately notified. The user then uses the terminal to operate the cooling system, referring to the suggested temperature adjustment plan, and returns the temperature to the specified range.

[0252] Examples of prompts to input into a generating AI model include, "Design a system that notifies when temperature data exceeds a set range," and "Describe an AI system that detects abnormal cell growth in real time and generates suggestions." These prompts help to show how the system operates under specific conditions.

[0253] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0254] Step 1:

[0255] The terminal uses measuring devices installed on the manufacturing line to collect environmental information in real time. Specifically, temperature and humidity sensors operate and continuously measure data. The input data collected includes physical conditions such as temperature, humidity, and cell growth rate. This data is converted into digital signals and immediately transmitted to the server. The output is a packet of digital data received by the server.

[0256] Step 2:

[0257] The server receives digital data transmitted from the terminal. Using the received data as input, it first performs preprocessing such as standardizing the data format and removing noise. For example, it might standardize units for temperature data or filter out outliers. This outputs a clean dataset for analysis. This dataset forms the basis for analysis by the AI ​​model.

[0258] Step 3:

[0259] The server inputs preprocessed data into a machine learning model for analysis. This analysis utilizes anomaly detection models based on TensorFlow or PyTorch. The analysis specifically involves learning data patterns and detecting unexpected fluctuations and anomalies. The output is the anomaly detection result, which provides information to determine whether or not an anomaly exists.

[0260] Step 4:

[0261] The server notifies the user through sensory means based on the anomaly detection results. Specifically, it displays a warning on the console screen or mobile device, notifying the user of the location and nature of the anomaly. For example, the user might be informed that "the temperature has exceeded the set value." The input is the anomaly detection result, and the output is the specific notification information for the user.

[0262] Step 5:

[0263] The user considers necessary adjustments based on notifications received through the terminal. The user confirms the system's suggestion (e.g., lower the cooling device temperature setting by 2 degrees) and performs the action. Specifically, the user controls the actuator and changes the setting by pressing the adjustment button on the terminal screen. The inputs are notification information and suggestions, and the output is the adjusted manufacturing conditions.

[0264] Step 6:

[0265] The server receives the adjustment results made by the user as feedback and stores them in a database. During this process, conditions for re-execution and new insights are accumulated. Specifically, the AI ​​model is adjusted and trained based on past feedback, improving the accuracy of subsequent analyses. The input to this process is user feedback, and the output is updated feedback data.

[0266] (Application Example 1)

[0267] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0268] As automation of monitoring and management progresses in factory manufacturing processes, real-time anomaly detection and immediate response are required. However, current systems suffer from low accuracy in anomaly detection and delayed responses, leading to decreased production efficiency and increased product defect rates. Therefore, a system is needed that can detect anomalies in real time with high accuracy and enable rapid and optimal responses.

[0269] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0270] In this invention, the server includes information acquisition means for real-time monitoring of the manufacturing process, analysis means including a machine learning algorithm for analyzing the acquired information, and communication means for detecting anomalies based on the analysis results and immediately notifying the administrator. This enables advanced anomaly detection, rapid notification, and response in real time.

[0271] An "information acquisition means" is a device that has the function of collecting environmental information such as temperature and humidity in real time using sensors during the manufacturing process within a factory.

[0272] A "machine learning algorithm" is an artificial intelligence technology used to analyze acquired data and detect or predict anomalies.

[0273] A "communication method" refers to a means of communication used to immediately notify administrators of anomalies detected through analysis and prompt them to take appropriate action.

[0274] A "suggestion system" is a system that has the function of providing the worker with the most suitable countermeasures for the detected anomaly.

[0275] "Implementation means" refers to equipment that carries out the proposed adjustments and operations in the manufacturing process and manages the process to ensure it continues properly.

[0276] A "user interface" is an interactive screen that visually displays anomaly detection information, allowing administrators and workers to easily understand the situation and take appropriate action.

[0277] The system that realizes this invention is designed to efficiently manage manufacturing processes in a factory. The server is equipped with information acquisition means that acquires information from the manufacturing line in real time using various sensors. These include temperature sensors, humidity sensors, and pH sensors. The acquired information is immediately transmitted to the server and analyzed using a Python-based machine learning algorithm. This analysis is performed using AI frameworks such as TensorFlow and PyTorch, and if an anomaly is detected, the administrator is notified by a communication means.

[0278] Users can visually confirm this anomaly through the provided user interface. This interface utilizes data visualization libraries such as Plotly to display the details of the anomaly and the proposed adjustments. Furthermore, the suggestion mechanism presents the optimal adjustment for the detected anomaly, allowing the user to manage the process based on it.

[0279] As a specific example, assume that the temperature exceeds the set range during cell culture in a certain factory. When the sensor detects a mutation, the information is sent to the server, analyzed by AI, and then notified to the administrator's terminal. At the same time, specific solutions for adjustment are displayed, and through the user's operation, the manufacturing environment can be optimized immediately.

[0280] Examples of prompt sentences include instructions considering specific scenarios such as "Develop an AI model for adjusting the production line when the temperature exceeds the appropriate range." In this way, advanced real-time manufacturing management and rapid response become possible.

[0281] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0282] Step 1:

[0283] The server acquires data in real time from various sensors installed in the factory. The sensors measure environmental data such as temperature, humidity, pH value, etc., and send the data to the server. The input is the measurement data of the sensors, and the data is aggregated to the server as the output.

[0284] Step 2:

[0285] The server analyzes the acquired data using machine learning algorithms. Utilize AI frameworks such as TensorFlow and PyTorch, and input the data into the anomaly detection model. The AI model compares with the set threshold value to identify anomalies. The input is the data from the sensors, and the analysis result is obtained as the output.

[0286] Step 3:

[0287] When an anomaly is detected, the server immediately sends a notification to the administrator via a communication channel. This notification includes details of the detected anomaly and its impact. The input is the result of AI analysis, and the output is a notification to the user. Specifically, the operation involves generating and sending an alert message.

[0288] Step 4:

[0289] The user receives notification content on the user interface and visually confirms the anomaly. Visualization libraries such as Plotly are used to display the anomaly patterns and timing. The input is a notification from the server, and the output is visualized data. Specifically, this involves viewing the data on the display.

[0290] Step 5:

[0291] The user reviews the proposed adjustments presented by the suggested method and selects the necessary actions. The suggestions include specific procedures for the optimal operation or configuration changes in response to an anomaly. The input is the proposed adjustments, and the output is the user's selection. The specific actions involve reviewing the adjustments and selecting actions.

[0292] Step 6:

[0293] The server optimizes the manufacturing process by performing adjustments using implementation methods based on user selections. This ensures that the manufacturing environment is properly maintained and product quality is preserved. The input is the user's adjustment instructions, and the output is the result of the performed adjustments. Specifically, this involves executing changes to the settings of the manufacturing line.

[0294] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0295] This invention provides a system that recognizes the user's emotional state and reflects it in the management of the manufacturing process in order to improve manufacturing efficiency and quality in the cell manufacturing process. Specifically, in addition to basic data acquisition means, analysis means, notification means, suggestion means, and execution means for managing the manufacturing process, it integrates an emotion engine that analyzes the user's emotions.

[0296] First, the terminal acquires various data from the manufacturing line and sends it to the server. The server uses artificial intelligence to analyze the data and detect anomalies. If an anomaly is detected, the information is immediately communicated to the user through a notification system.

[0297] This is where the emotion engine comes in. Installed on the server, the emotion engine analyzes the user's behavior and reactions when using the interface and infers the user's emotional state. For example, it can detect stress and anxiety from factors such as operation speed, error frequency, and facial expressions using facial recognition technology.

[0298] Subsequently, the emotion engine operates, along with suggestion methods, that are appropriate according to the user's emotions. If the user's stress level is determined to be high, the tone of the notification can be softened and the explanation can be made more detailed to reduce the user's burden. This allows the user to perform the adjustment work with peace of mind.

[0299] As a concrete example, if a user makes an error during the cell manufacturing process, the emotion engine detects the user's stress level. When the server notifies the user of the anomaly, it sends a notification to the user that includes an encouraging message based on the emotion engine's analysis. In this way, the user can maintain composure and smoothly adjust the manufacturing process. This system enables integrated management of the cell manufacturing process while taking the user's psychological state into consideration.

[0300] The following describes the processing flow.

[0301] Step 1:

[0302] The terminal collects data such as temperature, humidity, pH, and cell count in real time from sensors arranged throughout the production line. This data is used to monitor the production status.

[0303] Step 2:

[0304] The terminal transmits the collected data to the server. The server receives this data, stores it internally, and prepares for analysis.

[0305] Step 3:

[0306] The server analyzes the data using artificial intelligence. The AI executes an algorithm to detect anomalies by comparing with past data. Here, parameters outside the normal range are identified.

[0307] Step 4:

[0308] When the server detects an anomaly as a result of data analysis, it immediately communicates the abnormal situation to the user via the notification means. In so doing, include the type of anomaly and recommended countermeasures.

[0309] Step 5:

[0310] The server's emotion engine analyzes the user's emotional state based on data such as the user's operation history and the terminal camera. The AI may detect the user's stress, anxiety, etc.

[0311] Step 6:

[0312] The server adjusts the tone of the notification in consideration of the user's emotional state evaluated by the emotion engine. For example, if it is determined that the stress is high, it is possible to make the notification message concise and easy to understand and attach a motivating message.

[0313] Step 7:

[0314] Users check notifications sent from the server via their devices. These notifications include thoughtful instructions based on the analysis results of the emotion engine, allowing users to calmly choose the appropriate course of action.

[0315] Step 8:

[0316] The user implements the suggested adjustments on the terminal. The terminal applies the settings selected by the user to the production line and begins collecting new data.

[0317] Step 9:

[0318] The server will continue data analysis to confirm that the production line is operating normally in its adjusted state. This will help stabilize the manufacturing process.

[0319] (Example 2)

[0320] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0321] In manufacturing processes, real-time monitoring and immediate response to anomalies are crucial, but these are ineffective with conventional systems that do not adapt to the user's psychological state. In particular, appropriate responses are required in situations where users experience stress. Therefore, there is a need to provide a system that can respond based on the user's emotional state while maintaining the efficiency and quality of the manufacturing process.

[0322] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0323] In this invention, the server includes an information acquisition device means for monitoring the manufacturing process in real time, an analysis means including machine learning for analyzing the acquired information, and a means for using an emotion analysis device for analyzing the user's emotional state. This enables abnormal response and optimization of the manufacturing process that are tailored to the user's emotional state.

[0324] An "information acquisition device" is a device that collects data in real time using various sensors and measuring instruments in the manufacturing process.

[0325] "Machine learning" is a field of artificial intelligence that involves analyzing data and recognizing trends and patterns within it.

[0326] "Analysis methods" refer to the processes and techniques used to analyze acquired data and identify anomalies and trends.

[0327] A "notification device" is a device that has the function of notifying the user when an abnormality is detected.

[0328] A "suggestion device" is a system component that presents the user with the optimal countermeasure for an anomaly.

[0329] An "implementation device" is a device that reflects the conditions adjusted based on the proposal into the manufacturing process and has the function of monitoring the results again.

[0330] An "emotion analysis device" is a device that estimates a user's emotional state based on their behavior and facial expressions.

[0331] This invention is an integrated system that optimizes efficiency and quality in the manufacturing process and enables responses based on the user's psychological state. The system comprises an information acquisition device, an analysis means including machine learning, a notification device, a suggestion device, an implementation device, and an emotion analysis device that analyzes the user's emotional state.

[0332] First, the terminal acquires data in real time via sensors and devices installed on the manufacturing line and transmits it to a server. This data includes temperature, humidity, pressure, and operating status.

[0333] The server analyzes data acquired using a generative AI model and applies machine learning techniques to detect anomalies. This allows for the rapid identification of anomalies, ensuring that the manufacturing process is maintained optimally.

[0334] When an anomaly is detected, the server immediately notifies the user through an alerting device. In addition, an emotion analysis device analyzes the user's emotional state in real time, and uses the analysis results to adapt and adjust the tone of the notification and response measures. This allows for a response that minimizes the burden on the user, even when they are feeling stressed or anxious.

[0335] For example, if the temperature shows an abnormal value during the manufacturing process, the notification will include a prompt to check the cooling system, along with a message such as, "Please remain calm. We will provide you with detailed instructions."

[0336] A concrete example of a prompt message would be: "A data anomaly has been detected on the production line. The user may be feeling anxious. What reassuring message should be sent?"

[0337] In this way, the server and terminal work together to create a system that enables the smooth operation of the manufacturing process while taking into account the user's emotional state.

[0338] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0339] Step 1:

[0340] The terminal collects data from various sensors placed on the manufacturing line. Specifically, temperature, humidity, pressure, and machine operating status are acquired in real time. This input data is collected as basic information for monitoring the manufacturing process.

[0341] Step 2:

[0342] The terminal sends the collected data to the server. The data is transmitted via a communication protocol and delivered to the server in real time over the network. Through this process, the server obtains the data for analysis.

[0343] Step 3:

[0344] The server analyzes the received data using a generating AI model. Specifically, machine learning techniques are applied to detect anomaly patterns in the data. It analyzes real-time information as input data, and if an anomaly is detected, it outputs the result as an anomaly summary.

[0345] Step 4:

[0346] The server immediately notifies the user via an alerting device if an anomaly is detected. The notification includes an explanation of the anomaly, specific countermeasures, and encouraging or cautionary messages generated by a generative AI model. In this step, the notification content is adjusted based on the results of sentiment analysis, aiming to reduce the user's psychological burden.

[0347] Step 5:

[0348] Upon receiving a notification, the user receives guidance from the sentiment analysis device and makes appropriate adjustments. Based on the feedback from the sentiment analysis device, the server provides the next optimal prompts and suggestions to support the user. The user's behavior and responses are continuously monitored, and the sentiment data is used for further analysis.

[0349] This series of steps creates a system where servers and terminals work together to provide users with optimal guidance for the manufacturing process.

[0350] (Application Example 2)

[0351] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0352] Traditional manufacturing process management lacks consideration for the emotional state of users, leading to problems such as decreased work efficiency and quality. Furthermore, the lack of information provided to enable appropriate responses when anomalies are detected can cause users to experience excessive stress.

[0353] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0354] In this invention, the server includes means for acquiring information to perform real-time observation of the manufacturing process, means for analyzing the acquired information including machine learning, and means for integrating an emotion analysis engine to analyze the user's emotional state and provide optimal support according to the user's emotions. This enables efficient and high-quality management of the manufacturing process while taking the user's emotional state into consideration.

[0355] "Information acquisition means for real-time observation of the manufacturing process" refers to means for collecting necessary data in real time during manufacturing operations and processing it immediately.

[0356] "Analysis methods including machine learning to analyze acquired information" refers to methods that incorporate machine learning techniques to detect anomalies or patterns hidden within collected data.

[0357] "A means of integrating an emotion analysis engine that analyzes the user's emotional state and providing optimal support according to the user's emotions" refers to a means of evaluating the user's emotional state by analyzing their facial expressions and operation patterns, and then providing support accordingly.

[0358] The system for implementing this invention can acquire and analyze information during the manufacturing process and provide appropriate support tailored to the user's emotional state. The server uses information acquisition means to collect data from the manufacturing process in real time. This information is expected to be provided from terminals equipped with sensors and cameras. The acquired data is then analyzed by analysis means including machine learning. In this process, machine learning algorithms are used to detect known anomalies and patterns that can be improved.

[0359] Furthermore, an emotion analysis engine is integrated to analyze the user's emotional state. This engine analyzes behavioral data such as the user's facial expressions, operation speed, and error frequency, and uses this to infer the user's emotional state. The results of this analysis are used to provide the user with the best possible support. For example, if the user is feeling stressed, the notification tone may be softened and encouraging messages added to reduce the user's burden.

[0360] This system uses the OpenCV library and Keras to perform facial recognition and emotion analysis. Based on the analysis results, an SDK can be used for robot control to provide appropriate communication. For example, if a worker on a manufacturing line shows signs of fatigue, the emotion analysis engine can detect that emotion, and the robot can deliver a message such as, "You've worked hard. Why don't you take a short break?" An example of a prompt message to input into the AI ​​model generated based on this invention is, "Consider a factory robot system that analyzes an employee's facial expression to determine their emotion and provides optimal support if they are stressed."

[0361] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0362] Step 1:

[0363] The server receives data in real time from terminals equipped with sensors and cameras on the manufacturing line. This input data includes information about the progress of manufacturing and the facial expressions of workers. This data is first stored in a database and used as foundational data for subsequent analysis.

[0364] Step 2:

[0365] The server analyzes the acquired data using machine learning algorithms to detect signs of anomalies. This analysis includes data processing and classification to identify abnormal operating patterns and product quality degradation. The output generates detailed information about the detected anomalies.

[0366] Step 3:

[0367] The server uses an emotion analysis engine to analyze the user's emotional state. Input data includes the user's operation speed, error frequency, and facial expression data. Through data processing, this data is output as the user's stress and fatigue levels.

[0368] Step 4:

[0369] The server generates optimal support for the user based on anomaly notifications and analysis of emotional state. Specifically, it generates notifications in a gentle tone and sends them with encouraging messages. This process uses a generative AI model, which generates different messages depending on the prompt.

[0370] Step 5:

[0371] After the user receives a notification, the server monitors whether adjustments have been made in response to the notification and provides further support as needed. Throughout the entire process, the terminal and server exchange information in real time to optimize system operation.

[0372] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0373] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0374] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0375] [Third Embodiment]

[0376] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0377] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0378] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0379] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0380] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0381] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0382] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0383] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0384] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0385] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0386] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0387] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0388] This invention embodies a system for efficiently managing cell manufacturing processes and providing high-quality products. This system has the capability to monitor the manufacturing process in real time, detect anomalies, and propose appropriate countermeasures.

[0389] First, the terminal uses various sensors to acquire real-time data from the manufacturing line. This includes temperature, humidity, and cell growth rate, and this data is constantly transmitted to the server in an up-to-date state.

[0390] The server receives this data and immediately uses artificial intelligence to analyze it. The AI ​​employs learning algorithms to detect outliers and deviations from predictions. For example, it identifies anomalies when cell growth is slower than predicted or when environmental conditions fall outside the set range.

[0391] When an anomaly is detected, the server immediately transmits the information to the user through a notification system. For example, if the temperature exceeds the set range, the user receives a notification on their device and can review suggestions for adjusting the temperature.

[0392] The proposed adjustments are implemented in the manufacturing process in real time. Users can approve or adjust the optimal operations suggested by the AI ​​via a terminal. In this way, the entire manufacturing process is highly controlled, ensuring the production of uniform, high-quality products.

[0393] As a concrete example, let's assume a user is manufacturing a new batch of gene therapy drug. The terminal continuously generates and transmits manufacturing environment data, which is then analyzed by AI on the server. If there are no abnormalities, manufacturing proceeds without problems. However, if, for example, the pH becomes unstable, a notification is sent immediately, and adjustment suggestions are sent to the user. The user then makes the adjustments via the terminal, keeping the manufacturing process optimal. In this way, the entire system is automated, ensuring efficiency and product consistency.

[0394] The following describes the processing flow.

[0395] Step 1:

[0396] The terminal acquires real-time data such as temperature, humidity, pH, and cell count from sensors installed on the production line. This data is recorded at regular intervals and prepared for the next analysis step.

[0397] Step 2:

[0398] The terminal sequentially transmits the acquired data to the server. The server quickly receives the large amount of data and stores it in a database in preparation for analysis.

[0399] Step 3:

[0400] The server executes artificial intelligence based on the received data and begins analyzing it. During this process, the AI ​​detects anomalies by comparing the current data with past data and evaluates the deviation between the set baseline values ​​and the measured values.

[0401] Step 4:

[0402] When the server detects an anomaly based on the analysis results, it immediately generates a notification message and sends it to the user. This notification includes the location of the anomaly, the estimated cause, and recommended countermeasures.

[0403] Step 5:

[0404] The user receives a notification from the server via their device and confirms the nature of the anomaly. Simultaneously, the user has the opportunity to consider the proposed solutions.

[0405] Step 6:

[0406] The user approves and applies the proposed adjustments on the terminal. Once the user implements the adjustments, the terminal applies the new settings to the production line and monitors the process again.

[0407] Step 7:

[0408] The server continues to monitor real-time data even after adjustments are made, evaluating whether the manufacturing process is proceeding optimally. This ensures that the entire system is continuously optimized.

[0409] (Example 1)

[0410] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0411] In manufacturing processes, where real-time information gathering, anomaly detection, and rapid response are required, conventional systems have challenges in overall efficiency and accuracy because each process—data acquisition, analysis, notification, and proposal—is managed individually. Furthermore, the inability to respond quickly when anomalies occur can compromise product quality and process efficiency.

[0412] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0413] In this invention, the server includes a device means for collecting environmental information using measuring devices placed in the manufacturing process and transmitting it via communication means; an analysis means having a device for preprocessing the received information and analyzing it using a machine learning model; and a perception means for presenting anomalies detected from the analysis results using an output device and prompting human intervention. This enables integrated management of the entire manufacturing process, real-time anomaly detection, and rapid response.

[0414] A "manufacturing process" is a set of sequential tasks or procedures designed to produce an item.

[0415] A "measuring device" is a device used to measure physical or chemical quantities, and specifically includes equipment for acquiring environmental information such as temperature and humidity.

[0416] "Communication means" refers to a technology or method for exchanging information between different devices, enabling the transmission and reception of data.

[0417] "Analysis means" refers to technologies that have a process for processing and understanding collected data, and typically involve using computer programs to analyze the data.

[0418] A "machine learning model" is a set of algorithms that have the ability to learn from data and recognize patterns, and is particularly used to detect anomalies.

[0419] An "output device" is a collection of hardware that displays processed data and information in a format that humans can understand.

[0420] "Perceptual means" refers to functions and devices that facilitate human judgment and actions based on analyzed information.

[0421] A "corrective procedure" is a set of specific operations and steps taken to improve a detected problem or anomaly.

[0422] "Feedback" is the process of incorporating the results of operations and procedures performed back into the system to help with future operations and improvements.

[0423] This invention is an integrated system aimed at efficient management of the manufacturing process and improvement of quality. The system consists of a server, terminals, and users. The specific implementation will be described below.

[0424] The terminal's role is to collect environmental information using measuring devices placed on the manufacturing line. These measuring devices include temperature sensors, humidity sensors, and optical sensors, which are used to acquire real-time data. This data is transmitted to a server via communication means. The hardware utilizes state-of-the-art communication modules and sensors.

[0425] The server functions as a hub for efficiently processing received data. First, it preprocesses the data, and then performs data analysis using machine learning models. Frameworks such as TensorFlow and PyTorch are used for this analysis. Based on the analysis results, the server detects anomalies and presents this information to the user through perceptual means.

[0426] Users can make decisions based on the information presented and make adjustments according to the correction procedures suggested by the system. Users perform these adjustments via a terminal, optimizing various conditions in the manufacturing environment. The results are sent to the server as feedback and used to train the AI ​​model. This allows the server to improve the accuracy of anomaly detection in subsequent attempts.

[0427] As a concrete example, consider a scenario where a user is manufacturing a new drug. The terminal collects data such as temperature and pH level in real time and sends it to a server. The AI ​​on the server analyzes this data, and if it detects that the temperature exceeds the expected range, the user is immediately notified. The user then uses the terminal to operate the cooling system, referring to the suggested temperature adjustment plan, and returns the temperature to the specified range.

[0428] Examples of prompts to input into a generating AI model include, "Design a system that notifies when temperature data exceeds a set range," and "Describe an AI system that detects abnormal cell growth in real time and generates suggestions." These prompts help to show how the system operates under specific conditions.

[0429] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0430] Step 1:

[0431] The terminal uses measuring devices installed on the manufacturing line to collect environmental information in real time. Specifically, temperature and humidity sensors operate and continuously measure data. The input data collected includes physical conditions such as temperature, humidity, and cell growth rate. This data is converted into digital signals and immediately transmitted to the server. The output is a packet of digital data received by the server.

[0432] Step 2:

[0433] The server receives digital data transmitted from the terminal. Using the received data as input, it first performs preprocessing such as standardizing the data format and removing noise. For example, it might standardize units for temperature data or filter out outliers. This outputs a clean dataset for analysis. This dataset forms the basis for analysis by the AI ​​model.

[0434] Step 3:

[0435] The server inputs preprocessed data into a machine learning model for analysis. This analysis utilizes anomaly detection models based on TensorFlow or PyTorch. The analysis specifically involves learning data patterns and detecting unexpected fluctuations and anomalies. The output is the anomaly detection result, which provides information to determine whether or not an anomaly exists.

[0436] Step 4:

[0437] The server notifies the user through sensory means based on the anomaly detection results. Specifically, it displays a warning on the console screen or mobile device, notifying the user of the location and nature of the anomaly. For example, the user might be informed that "the temperature has exceeded the set value." The input is the anomaly detection result, and the output is the specific notification information for the user.

[0438] Step 5:

[0439] The user considers necessary adjustments based on notifications received through the terminal. The user confirms the system's suggestion (e.g., lower the cooling device temperature setting by 2 degrees) and performs the action. Specifically, the user controls the actuator and changes the setting by pressing the adjustment button on the terminal screen. The inputs are notification information and suggestions, and the output is the adjusted manufacturing conditions.

[0440] Step 6:

[0441] The server receives the adjustment results made by the user as feedback and stores them in a database. During this process, conditions for re-execution and new insights are accumulated. Specifically, the AI ​​model is adjusted and trained based on past feedback, improving the accuracy of subsequent analyses. The input to this process is user feedback, and the output is updated feedback data.

[0442] (Application Example 1)

[0443] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0444] As automation of monitoring and management progresses in factory manufacturing processes, real-time anomaly detection and immediate response are required. However, current systems suffer from low accuracy in anomaly detection and delayed responses, leading to decreased production efficiency and increased product defect rates. Therefore, a system is needed that can detect anomalies in real time with high accuracy and enable rapid and optimal responses.

[0445] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0446] In this invention, the server includes information acquisition means for real-time monitoring of the manufacturing process, analysis means including a machine learning algorithm for analyzing the acquired information, and communication means for detecting anomalies based on the analysis results and immediately notifying the administrator. This enables advanced anomaly detection, rapid notification, and response in real time.

[0447] An "information acquisition means" is a device that has the function of collecting environmental information such as temperature and humidity in real time using sensors during the manufacturing process within a factory.

[0448] A "machine learning algorithm" is an artificial intelligence technology used to analyze acquired data and detect or predict anomalies.

[0449] A "communication method" refers to a means of communication used to immediately notify administrators of anomalies detected through analysis and prompt them to take appropriate action.

[0450] A "suggestion system" is a system that has the function of providing the worker with the most suitable countermeasures for the detected anomaly.

[0451] "Implementation means" refers to equipment that carries out the proposed adjustments and operations in the manufacturing process and manages the process to ensure it continues properly.

[0452] A "user interface" is an interactive screen that visually displays anomaly detection information, allowing administrators and workers to easily understand the situation and take appropriate action.

[0453] The system that realizes this invention is designed to efficiently manage manufacturing processes in a factory. The server is equipped with information acquisition means that acquires information from the manufacturing line in real time using various sensors. These include temperature sensors, humidity sensors, and pH sensors. The acquired information is immediately transmitted to the server and analyzed using a Python-based machine learning algorithm. This analysis is performed using AI frameworks such as TensorFlow and PyTorch, and if an anomaly is detected, the administrator is notified by a communication means.

[0454] Users can visually confirm this anomaly through the provided user interface. This interface utilizes data visualization libraries such as Plotly to display the details of the anomaly and the proposed adjustments. Furthermore, the suggestion mechanism presents the optimal adjustment for the detected anomaly, allowing the user to manage the process based on it.

[0455] As a concrete example, consider a scenario where the temperature exceeds a set range during cell culture in a factory. When a sensor detects a mutation, the information is sent to a server, analyzed by AI, and then notified to the administrator's terminal. Simultaneously, specific adjustment suggestions are displayed, and the manufacturing environment is immediately optimized through user intervention.

[0456] Examples of prompt statements include commands that consider specific scenarios, such as, "Develop an AI model to adjust the production line if the temperature exceeds the appropriate range." In this way, advanced real-time production management and rapid response become possible.

[0457] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0458] Step 1:

[0459] The server acquires data in real time from various sensors installed within the factory. The sensors measure environmental data such as temperature, humidity, and pH values, and transmit this data to the server. The input is the measurement data from the sensors, and the output is aggregated on the server.

[0460] Step 2:

[0461] The server analyzes the acquired data using machine learning algorithms. It utilizes AI frameworks such as TensorFlow and PyTorch to input data into an anomaly detection model. The AI ​​model identifies anomalies by comparing the data against set thresholds. The input is data from sensors, and the output is the analysis results.

[0462] Step 3:

[0463] When an anomaly is detected, the server immediately sends a notification to the administrator via a communication channel. This notification includes details of the detected anomaly and its impact. The input is the result of AI analysis, and the output is a notification to the user. Specifically, the operation involves generating and sending an alert message.

[0464] Step 4:

[0465] The user receives notification content on the user interface and visually confirms the anomaly. Visualization libraries such as Plotly are used to display the anomaly patterns and timing. The input is a notification from the server, and the output is visualized data. Specifically, this involves viewing the data on the display.

[0466] Step 5:

[0467] The user reviews the proposed adjustments presented by the suggested method and selects the necessary actions. The suggestions include specific procedures for the optimal operation or configuration changes in response to an anomaly. The input is the proposed adjustments, and the output is the user's selection. The specific actions involve reviewing the adjustments and selecting actions.

[0468] Step 6:

[0469] The server optimizes the manufacturing process by performing adjustments using implementation methods based on user selections. This ensures that the manufacturing environment is properly maintained and product quality is preserved. The input is the user's adjustment instructions, and the output is the result of the performed adjustments. Specifically, this involves executing changes to the settings of the manufacturing line.

[0470] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0471] This invention provides a system that recognizes the user's emotional state and reflects it in the management of the manufacturing process in order to improve manufacturing efficiency and quality in the cell manufacturing process. Specifically, in addition to basic data acquisition means, analysis means, notification means, suggestion means, and execution means for managing the manufacturing process, it integrates an emotion engine that analyzes the user's emotions.

[0472] First, the terminal acquires various data from the manufacturing line and sends it to the server. The server uses artificial intelligence to analyze the data and detect anomalies. If an anomaly is detected, the information is immediately communicated to the user through a notification system.

[0473] This is where the emotion engine comes in. Installed on the server, the emotion engine analyzes the user's behavior and reactions when using the interface and infers the user's emotional state. For example, it can detect stress and anxiety from factors such as operation speed, error frequency, and facial expressions using facial recognition technology.

[0474] Subsequently, the emotion engine operates, along with suggestion methods, that are appropriate according to the user's emotions. If the user's stress level is determined to be high, the tone of the notification can be softened and the explanation can be made more detailed to reduce the user's burden. This allows the user to perform the adjustment work with peace of mind.

[0475] As a concrete example, if a user makes an error during the cell manufacturing process, the emotion engine detects the user's stress level. When the server notifies the user of the anomaly, it sends a notification to the user that includes an encouraging message based on the emotion engine's analysis. In this way, the user can maintain composure and smoothly adjust the manufacturing process. This system enables integrated management of the cell manufacturing process while taking the user's psychological state into consideration.

[0476] The following describes the processing flow.

[0477] Step 1:

[0478] The terminal collects data such as temperature, humidity, pH, and cell count in real time from sensors placed throughout the production line. This data is used to monitor the production conditions.

[0479] Step 2:

[0480] The terminal sends the collected data to the server. The server receives this data, stores it internally, and prepares it for analysis.

[0481] Step 3:

[0482] The server uses artificial intelligence to analyze the data. The AI ​​executes an algorithm that detects anomalies by comparing the data with past data. Here, it identifies parameters that fall outside the normal range.

[0483] Step 4:

[0484] When the server detects an anomaly as a result of data analysis, it immediately notifies the user of the anomaly via a notification system. This notification includes the type of anomaly and recommended countermeasures.

[0485] Step 5:

[0486] The server's emotion engine analyzes the user's emotional state based on data such as the user's operation history and device camera. The AI ​​may detect the user's stress, anxiety, and other emotional states.

[0487] Step 6:

[0488] The server takes into account the user's emotional state, as assessed by the emotion engine, and adjusts the tone of notifications accordingly. For example, if the user is judged to be highly stressed, the notification message can be made concise and easy to understand, and may include an encouraging message.

[0489] Step 7:

[0490] Users check notifications sent from the server via their devices. These notifications include thoughtful instructions based on the analysis results of the emotion engine, allowing users to calmly choose the appropriate course of action.

[0491] Step 8:

[0492] The user implements the suggested adjustments on the terminal. The terminal applies the settings selected by the user to the production line and begins collecting new data.

[0493] Step 9:

[0494] The server will continue data analysis to confirm that the production line is operating normally in its adjusted state. This will help stabilize the manufacturing process.

[0495] (Example 2)

[0496] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0497] In manufacturing processes, real-time monitoring and immediate response to anomalies are crucial, but these are ineffective with conventional systems that do not adapt to the user's psychological state. In particular, appropriate responses are required in situations where users experience stress. Therefore, there is a need to provide a system that can respond based on the user's emotional state while maintaining the efficiency and quality of the manufacturing process.

[0498] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0499] In this invention, the server includes an information acquisition device means for monitoring the manufacturing process in real time, an analysis means including machine learning for analyzing the acquired information, and a means for using an emotion analysis device for analyzing the user's emotional state. This enables abnormal response and optimization of the manufacturing process that are tailored to the user's emotional state.

[0500] An "information acquisition device" is a device that collects data in real time using various sensors and measuring instruments in the manufacturing process.

[0501] "Machine learning" is a field of artificial intelligence that involves analyzing data and recognizing trends and patterns within it.

[0502] "Analysis methods" refer to the processes and techniques used to analyze acquired data and identify anomalies and trends.

[0503] A "notification device" is a device that has the function of notifying the user when an abnormality is detected.

[0504] A "suggestion device" is a system component that presents the user with the optimal countermeasure for an anomaly.

[0505] An "implementation device" is a device that reflects the conditions adjusted based on the proposal into the manufacturing process and has the function of monitoring the results again.

[0506] An "emotion analysis device" is a device that estimates a user's emotional state based on their behavior and facial expressions.

[0507] This invention is an integrated system that optimizes efficiency and quality in the manufacturing process and enables responses based on the user's psychological state. The system comprises an information acquisition device, an analysis means including machine learning, a notification device, a suggestion device, an implementation device, and an emotion analysis device that analyzes the user's emotional state.

[0508] First, the terminal acquires data in real time via sensors and devices installed on the manufacturing line and transmits it to a server. This data includes temperature, humidity, pressure, and operating status.

[0509] The server analyzes data acquired using a generative AI model and applies machine learning techniques to detect anomalies. This allows for the rapid identification of anomalies, ensuring that the manufacturing process is maintained optimally.

[0510] When an anomaly is detected, the server immediately notifies the user through an alerting device. In addition, an emotion analysis device analyzes the user's emotional state in real time, and uses the analysis results to adapt and adjust the tone of the notification and response measures. This allows for a response that minimizes the burden on the user, even when they are feeling stressed or anxious.

[0511] For example, if the temperature shows an abnormal value during the manufacturing process, the notification will include a prompt to check the cooling system, along with a message such as, "Please remain calm. We will provide you with detailed instructions."

[0512] A concrete example of a prompt message would be: "A data anomaly has been detected on the production line. The user may be feeling anxious. What reassuring message should be sent?"

[0513] In this way, the server and terminal work together to create a system that enables the smooth operation of the manufacturing process while taking into account the user's emotional state.

[0514] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0515] Step 1:

[0516] The terminal collects data from various sensors placed on the manufacturing line. Specifically, temperature, humidity, pressure, and machine operating status are acquired in real time. This input data is collected as basic information for monitoring the manufacturing process.

[0517] Step 2:

[0518] The terminal sends the collected data to the server. The data is transmitted via a communication protocol and delivered to the server in real time over the network. Through this process, the server obtains the data for analysis.

[0519] Step 3:

[0520] The server analyzes the received data using a generating AI model. Specifically, machine learning techniques are applied to detect anomaly patterns in the data. It analyzes real-time information as input data, and if an anomaly is detected, it outputs the result as an anomaly summary.

[0521] Step 4:

[0522] The server immediately notifies the user via an alerting device if an anomaly is detected. The notification includes an explanation of the anomaly, specific countermeasures, and encouraging or cautionary messages generated by a generative AI model. In this step, the notification content is adjusted based on the results of sentiment analysis, aiming to reduce the user's psychological burden.

[0523] Step 5:

[0524] Upon receiving a notification, the user receives guidance from the sentiment analysis device and makes appropriate adjustments. Based on the feedback from the sentiment analysis device, the server provides the next optimal prompts and suggestions to support the user. The user's behavior and responses are continuously monitored, and the sentiment data is used for further analysis.

[0525] This series of steps creates a system where servers and terminals work together to provide users with optimal guidance for the manufacturing process.

[0526] (Application Example 2)

[0527] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0528] Traditional manufacturing process management lacks consideration for the emotional state of users, leading to problems such as decreased work efficiency and quality. Furthermore, the lack of information provided to enable appropriate responses when anomalies are detected can cause users to experience excessive stress.

[0529] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0530] In this invention, the server includes means for acquiring information to perform real-time observation of the manufacturing process, means for analyzing the acquired information including machine learning, and means for integrating an emotion analysis engine to analyze the user's emotional state and provide optimal support according to the user's emotions. This enables efficient and high-quality management of the manufacturing process while taking the user's emotional state into consideration.

[0531] "Information acquisition means for real-time observation of the manufacturing process" refers to means for collecting necessary data in real time during manufacturing operations and processing it immediately.

[0532] "Analysis methods including machine learning to analyze acquired information" refers to methods that incorporate machine learning techniques to detect anomalies or patterns hidden within collected data.

[0533] "A means of integrating an emotion analysis engine that analyzes the user's emotional state and providing optimal support according to the user's emotions" refers to a means of evaluating the user's emotional state by analyzing their facial expressions and operation patterns, and then providing support accordingly.

[0534] The system for implementing this invention can acquire and analyze information during the manufacturing process and provide appropriate support tailored to the user's emotional state. The server uses information acquisition means to collect data from the manufacturing process in real time. This information is expected to be provided from terminals equipped with sensors and cameras. The acquired data is then analyzed by analysis means including machine learning. In this process, machine learning algorithms are used to detect known anomalies and patterns that can be improved.

[0535] Furthermore, an emotion analysis engine is integrated to analyze the user's emotional state. This engine analyzes behavioral data such as the user's facial expressions, operation speed, and error frequency, and uses this to infer the user's emotional state. The results of this analysis are used to provide the user with the best possible support. For example, if the user is feeling stressed, the notification tone may be softened and encouraging messages added to reduce the user's burden.

[0536] This system uses the OpenCV library and Keras to perform facial recognition and emotion analysis. Based on the analysis results, an SDK can be used for robot control to provide appropriate communication. For example, if a worker on a manufacturing line shows signs of fatigue, the emotion analysis engine can detect that emotion, and the robot can deliver a message such as, "You've worked hard. Why don't you take a short break?" An example of a prompt message to input into the AI ​​model generated based on this invention is, "Consider a factory robot system that analyzes an employee's facial expression to determine their emotion and provides optimal support if they are stressed."

[0537] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0538] Step 1:

[0539] The server receives data in real time from terminals equipped with sensors and cameras on the manufacturing line. This input data includes information about the progress of manufacturing and the facial expressions of workers. This data is first stored in a database and used as foundational data for subsequent analysis.

[0540] Step 2:

[0541] The server analyzes the acquired data using machine learning algorithms to detect signs of anomalies. This analysis includes data processing and classification to identify abnormal operating patterns and product quality degradation. The output generates detailed information about the detected anomalies.

[0542] Step 3:

[0543] The server uses an emotion analysis engine to analyze the user's emotional state. Input data includes the user's operation speed, error frequency, and facial expression data. Through data processing, this data is output as the user's stress and fatigue levels.

[0544] Step 4:

[0545] The server generates optimal support for the user based on anomaly notifications and analysis of emotional state. Specifically, it generates notifications in a gentle tone and sends them with encouraging messages. This process uses a generative AI model, which generates different messages depending on the prompt.

[0546] Step 5:

[0547] After the user receives a notification, the server monitors whether adjustments have been made in response to the notification and provides further support as needed. Throughout the entire process, the terminal and server exchange information in real time to optimize system operation.

[0548] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0549] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0550] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0551] [Fourth Embodiment]

[0552] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0553] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0554] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0555] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0556] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0557] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0558] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0559] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0560] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0561] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0562] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0563] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0564] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0565] This invention embodies a system for efficiently managing cell manufacturing processes and providing high-quality products. This system has the capability to monitor the manufacturing process in real time, detect anomalies, and propose appropriate countermeasures.

[0566] First, the terminal uses various sensors to acquire real-time data from the manufacturing line. This includes temperature, humidity, and cell growth rate, and this data is constantly transmitted to the server in an up-to-date state.

[0567] The server receives this data and immediately uses artificial intelligence to analyze it. The AI ​​employs learning algorithms to detect outliers and deviations from predictions. For example, it identifies anomalies when cell growth is slower than predicted or when environmental conditions fall outside the set range.

[0568] When an anomaly is detected, the server immediately transmits the information to the user through a notification system. For example, if the temperature exceeds the set range, the user receives a notification on their device and can review suggestions for adjusting the temperature.

[0569] The proposed adjustments are implemented in the manufacturing process in real time. Users can approve or adjust the optimal operations suggested by the AI ​​via a terminal. In this way, the entire manufacturing process is highly controlled, ensuring the production of uniform, high-quality products.

[0570] As a concrete example, let's assume a user is manufacturing a new batch of gene therapy drug. The terminal continuously generates and transmits manufacturing environment data, which is then analyzed by AI on the server. If there are no abnormalities, manufacturing proceeds without problems. However, if, for example, the pH becomes unstable, a notification is sent immediately, and adjustment suggestions are sent to the user. The user then makes the adjustments via the terminal, keeping the manufacturing process optimal. In this way, the entire system is automated, ensuring efficiency and product consistency.

[0571] The following describes the processing flow.

[0572] Step 1:

[0573] The terminal acquires real-time data such as temperature, humidity, pH, and cell count from sensors installed on the production line. This data is recorded at regular intervals and prepared for the next analysis step.

[0574] Step 2:

[0575] The terminal sequentially transmits the acquired data to the server. The server quickly receives the large amount of data and stores it in a database in preparation for analysis.

[0576] Step 3:

[0577] The server executes artificial intelligence based on the received data and begins analyzing it. During this process, the AI ​​detects anomalies by comparing the current data with past data and evaluates the deviation between the set baseline values ​​and the measured values.

[0578] Step 4:

[0579] When the server detects an anomaly based on the analysis results, it immediately generates a notification message and sends it to the user. This notification includes the location of the anomaly, the estimated cause, and recommended countermeasures.

[0580] Step 5:

[0581] The user receives a notification from the server via their device and confirms the nature of the anomaly. Simultaneously, the user has the opportunity to consider the proposed solutions.

[0582] Step 6:

[0583] The user approves and applies the proposed adjustments on the terminal. Once the user implements the adjustments, the terminal applies the new settings to the production line and monitors the process again.

[0584] Step 7:

[0585] The server continues to monitor real-time data even after adjustments are made, evaluating whether the manufacturing process is proceeding optimally. This ensures that the entire system is continuously optimized.

[0586] (Example 1)

[0587] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0588] In manufacturing processes, where real-time information gathering, anomaly detection, and rapid response are required, conventional systems have challenges in overall efficiency and accuracy because each process—data acquisition, analysis, notification, and proposal—is managed individually. Furthermore, the inability to respond quickly when anomalies occur can compromise product quality and process efficiency.

[0589] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0590] In this invention, the server includes a device means for collecting environmental information using measuring devices placed in the manufacturing process and transmitting it via communication means; an analysis means having a device for preprocessing the received information and analyzing it using a machine learning model; and a perception means for presenting anomalies detected from the analysis results using an output device and prompting human intervention. This enables integrated management of the entire manufacturing process, real-time anomaly detection, and rapid response.

[0591] A "manufacturing process" is a set of sequential tasks or procedures designed to produce an item.

[0592] A "measuring device" is a device used to measure physical or chemical quantities, and specifically includes equipment for acquiring environmental information such as temperature and humidity.

[0593] "Communication means" refers to a technology or method for exchanging information between different devices, enabling the transmission and reception of data.

[0594] "Analysis means" refers to technologies that have a process for processing and understanding collected data, and typically involve using computer programs to analyze the data.

[0595] A "machine learning model" is a set of algorithms that have the ability to learn from data and recognize patterns, and is particularly used to detect anomalies.

[0596] An "output device" is a collection of hardware that displays processed data and information in a format that humans can understand.

[0597] "Perceptual means" refers to functions and devices that facilitate human judgment and actions based on analyzed information.

[0598] A "corrective procedure" is a set of specific operations and steps taken to improve a detected problem or anomaly.

[0599] "Feedback" is the process of incorporating the results of operations and procedures performed back into the system to help with future operations and improvements.

[0600] This invention is an integrated system aimed at efficient management of the manufacturing process and improvement of quality. The system consists of a server, terminals, and users. The specific implementation will be described below.

[0601] The terminal's role is to collect environmental information using measuring devices placed on the manufacturing line. These measuring devices include temperature sensors, humidity sensors, and optical sensors, which are used to acquire real-time data. This data is transmitted to a server via communication means. The hardware utilizes state-of-the-art communication modules and sensors.

[0602] The server functions as a hub for efficiently processing received data. First, it preprocesses the data, and then performs data analysis using machine learning models. Frameworks such as TensorFlow and PyTorch are used for this analysis. Based on the analysis results, the server detects anomalies and presents this information to the user through perceptual means.

[0603] Users can make decisions based on the information presented and make adjustments according to the correction procedures suggested by the system. Users perform these adjustments via a terminal, optimizing various conditions in the manufacturing environment. The results are sent to the server as feedback and used to train the AI ​​model. This allows the server to improve the accuracy of anomaly detection in subsequent attempts.

[0604] As a concrete example, consider a scenario where a user is manufacturing a new drug. The terminal collects data such as temperature and pH level in real time and sends it to a server. The AI ​​on the server analyzes this data, and if it detects that the temperature exceeds the expected range, the user is immediately notified. The user then uses the terminal to operate the cooling system, referring to the suggested temperature adjustment plan, and returns the temperature to the specified range.

[0605] Examples of prompts to input into a generating AI model include, "Design a system that notifies when temperature data exceeds a set range," and "Describe an AI system that detects abnormal cell growth in real time and generates suggestions." These prompts help to show how the system operates under specific conditions.

[0606] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0607] Step 1:

[0608] The terminal uses measuring devices installed on the manufacturing line to collect environmental information in real time. Specifically, temperature and humidity sensors operate and continuously measure data. The input data collected includes physical conditions such as temperature, humidity, and cell growth rate. This data is converted into digital signals and immediately transmitted to the server. The output is a packet of digital data received by the server.

[0609] Step 2:

[0610] The server receives digital data transmitted from the terminal. Using the received data as input, it first performs preprocessing such as standardizing the data format and removing noise. For example, it might standardize units for temperature data or filter out outliers. This outputs a clean dataset for analysis. This dataset forms the basis for analysis by the AI ​​model.

[0611] Step 3:

[0612] The server inputs preprocessed data into a machine learning model for analysis. This analysis utilizes anomaly detection models based on TensorFlow or PyTorch. The analysis specifically involves learning data patterns and detecting unexpected fluctuations and anomalies. The output is the anomaly detection result, which provides information to determine whether or not an anomaly exists.

[0613] Step 4:

[0614] The server notifies the user through sensory means based on the anomaly detection results. Specifically, it displays a warning on the console screen or mobile device, notifying the user of the location and nature of the anomaly. For example, the user might be informed that "the temperature has exceeded the set value." The input is the anomaly detection result, and the output is the specific notification information for the user.

[0615] Step 5:

[0616] The user considers necessary adjustments based on notifications received through the terminal. The user confirms the system's suggestion (e.g., lower the cooling device temperature setting by 2 degrees) and performs the action. Specifically, the user controls the actuator and changes the setting by pressing the adjustment button on the terminal screen. The inputs are notification information and suggestions, and the output is the adjusted manufacturing conditions.

[0617] Step 6:

[0618] The server receives the adjustment results made by the user as feedback and stores them in a database. During this process, conditions for re-execution and new insights are accumulated. Specifically, the AI ​​model is adjusted and trained based on past feedback, improving the accuracy of subsequent analyses. The input to this process is user feedback, and the output is updated feedback data.

[0619] (Application Example 1)

[0620] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] As automation of monitoring and management progresses in factory manufacturing processes, real-time anomaly detection and immediate response are required. However, current systems suffer from low accuracy in anomaly detection and delayed responses, leading to decreased production efficiency and increased product defect rates. Therefore, a system is needed that can detect anomalies in real time with high accuracy and enable rapid and optimal responses.

[0622] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0623] In this invention, the server includes information acquisition means for real-time monitoring of the manufacturing process, analysis means including a machine learning algorithm for analyzing the acquired information, and communication means for detecting anomalies based on the analysis results and immediately notifying the administrator. This enables advanced anomaly detection, rapid notification, and response in real time.

[0624] An "information acquisition means" is a device that has the function of collecting environmental information such as temperature and humidity in real time using sensors during the manufacturing process within a factory.

[0625] A "machine learning algorithm" is an artificial intelligence technology used to analyze acquired data and detect or predict anomalies.

[0626] A "communication method" refers to a means of communication used to immediately notify administrators of anomalies detected through analysis and prompt them to take appropriate action.

[0627] A "suggestion system" is a system that has the function of providing the worker with the most suitable countermeasures for the detected anomaly.

[0628] "Implementation means" refers to equipment that carries out the proposed adjustments and operations in the manufacturing process and manages the process to ensure it continues properly.

[0629] A "user interface" is an interactive screen that visually displays anomaly detection information, allowing administrators and workers to easily understand the situation and take appropriate action.

[0630] The system that realizes this invention is designed to efficiently manage manufacturing processes in a factory. The server is equipped with information acquisition means that acquires information from the manufacturing line in real time using various sensors. These include temperature sensors, humidity sensors, and pH sensors. The acquired information is immediately transmitted to the server and analyzed using a Python-based machine learning algorithm. This analysis is performed using AI frameworks such as TensorFlow and PyTorch, and if an anomaly is detected, the administrator is notified by a communication means.

[0631] Users can visually confirm this anomaly through the provided user interface. This interface utilizes data visualization libraries such as Plotly to display the details of the anomaly and the proposed adjustments. Furthermore, the suggestion mechanism presents the optimal adjustment for the detected anomaly, allowing the user to manage the process based on it.

[0632] As a concrete example, consider a scenario where the temperature exceeds a set range during cell culture in a factory. When a sensor detects a mutation, the information is sent to a server, analyzed by AI, and then notified to the administrator's terminal. Simultaneously, specific adjustment suggestions are displayed, and the manufacturing environment is immediately optimized through user intervention.

[0633] Examples of prompt statements include commands that consider specific scenarios, such as, "Develop an AI model to adjust the production line if the temperature exceeds the appropriate range." In this way, advanced real-time production management and rapid response become possible.

[0634] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0635] Step 1:

[0636] The server acquires data in real time from various sensors installed within the factory. The sensors measure environmental data such as temperature, humidity, and pH values, and transmit this data to the server. The input is the measurement data from the sensors, and the output is aggregated on the server.

[0637] Step 2:

[0638] The server analyzes the acquired data using machine learning algorithms. It utilizes AI frameworks such as TensorFlow and PyTorch to input data into an anomaly detection model. The AI ​​model identifies anomalies by comparing the data against set thresholds. The input is data from sensors, and the output is the analysis results.

[0639] Step 3:

[0640] When an anomaly is detected, the server immediately sends a notification to the administrator via a communication channel. This notification includes details of the detected anomaly and its impact. The input is the result of AI analysis, and the output is a notification to the user. Specifically, the operation involves generating and sending an alert message.

[0641] Step 4:

[0642] The user receives notification content on the user interface and visually confirms the anomaly. Visualization libraries such as Plotly are used to display the anomaly patterns and timing. The input is a notification from the server, and the output is visualized data. Specifically, this involves viewing the data on the display.

[0643] Step 5:

[0644] The user reviews the proposed adjustments presented by the suggested method and selects the necessary actions. The suggestions include specific procedures for the optimal operation or configuration changes in response to an anomaly. The input is the proposed adjustments, and the output is the user's selection. The specific actions involve reviewing the adjustments and selecting actions.

[0645] Step 6:

[0646] The server optimizes the manufacturing process by performing adjustments using implementation methods based on user selections. This ensures that the manufacturing environment is properly maintained and product quality is preserved. The input is the user's adjustment instructions, and the output is the result of the performed adjustments. Specifically, this involves executing changes to the settings of the manufacturing line.

[0647] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0648] This invention provides a system that recognizes the user's emotional state and reflects it in the management of the manufacturing process in order to improve manufacturing efficiency and quality in the cell manufacturing process. Specifically, in addition to basic data acquisition means, analysis means, notification means, suggestion means, and execution means for managing the manufacturing process, it integrates an emotion engine that analyzes the user's emotions.

[0649] First, the terminal acquires various data from the manufacturing line and sends it to the server. The server uses artificial intelligence to analyze the data and detect anomalies. If an anomaly is detected, the information is immediately communicated to the user through a notification system.

[0650] This is where the emotion engine comes in. Installed on the server, the emotion engine analyzes the user's behavior and reactions when using the interface and infers the user's emotional state. For example, it can detect stress and anxiety from factors such as operation speed, error frequency, and facial expressions using facial recognition technology.

[0651] Subsequently, the emotion engine operates, along with suggestion methods, that are appropriate according to the user's emotions. If the user's stress level is determined to be high, the tone of the notification can be softened and the explanation can be made more detailed to reduce the user's burden. This allows the user to perform the adjustment work with peace of mind.

[0652] As a concrete example, if a user makes an error during the cell manufacturing process, the emotion engine detects the user's stress level. When the server notifies the user of the anomaly, it sends a notification to the user that includes an encouraging message based on the emotion engine's analysis. In this way, the user can maintain composure and smoothly adjust the manufacturing process. This system enables integrated management of the cell manufacturing process while taking the user's psychological state into consideration.

[0653] The following describes the processing flow.

[0654] Step 1:

[0655] The terminal collects data such as temperature, humidity, pH, and cell count in real time from sensors placed throughout the production line. This data is used to monitor the production conditions.

[0656] Step 2:

[0657] The terminal sends the collected data to the server. The server receives this data, stores it internally, and prepares it for analysis.

[0658] Step 3:

[0659] The server uses artificial intelligence to analyze the data. The AI ​​executes an algorithm that detects anomalies by comparing the data with past data. Here, it identifies parameters that fall outside the normal range.

[0660] Step 4:

[0661] When the server detects an anomaly as a result of data analysis, it immediately notifies the user of the anomaly via a notification system. This notification includes the type of anomaly and recommended countermeasures.

[0662] Step 5:

[0663] The server's emotion engine analyzes the user's emotional state based on data such as the user's operation history and device camera. The AI ​​may detect the user's stress, anxiety, and other emotional states.

[0664] Step 6:

[0665] The server takes into account the user's emotional state, as assessed by the emotion engine, and adjusts the tone of notifications accordingly. For example, if the user is judged to be highly stressed, the notification message can be made concise and easy to understand, and may include an encouraging message.

[0666] Step 7:

[0667] Users check notifications sent from the server via their devices. These notifications include thoughtful instructions based on the analysis results of the emotion engine, allowing users to calmly choose the appropriate course of action.

[0668] Step 8:

[0669] The user implements the suggested adjustments on the terminal. The terminal applies the settings selected by the user to the production line and begins collecting new data.

[0670] Step 9:

[0671] The server will continue data analysis to confirm that the production line is operating normally in its adjusted state. This will help stabilize the manufacturing process.

[0672] (Example 2)

[0673] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0674] In manufacturing processes, real-time monitoring and immediate response to anomalies are crucial, but these are ineffective with conventional systems that do not adapt to the user's psychological state. In particular, appropriate responses are required in situations where users experience stress. Therefore, there is a need to provide a system that can respond based on the user's emotional state while maintaining the efficiency and quality of the manufacturing process.

[0675] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0676] In this invention, the server includes an information acquisition device means for monitoring the manufacturing process in real time, an analysis means including machine learning for analyzing the acquired information, and a means for using an emotion analysis device for analyzing the user's emotional state. This enables abnormal response and optimization of the manufacturing process that are tailored to the user's emotional state.

[0677] An "information acquisition device" is a device that collects data in real time using various sensors and measuring instruments in the manufacturing process.

[0678] "Machine learning" is a field of artificial intelligence that involves analyzing data and recognizing trends and patterns within it.

[0679] "Analysis methods" refer to the processes and techniques used to analyze acquired data and identify anomalies and trends.

[0680] A "notification device" is a device that has the function of notifying the user when an abnormality is detected.

[0681] A "suggestion device" is a system component that presents the user with the optimal countermeasure for an anomaly.

[0682] An "implementation device" is a device that reflects the conditions adjusted based on the proposal into the manufacturing process and has the function of monitoring the results again.

[0683] An "emotion analysis device" is a device that estimates a user's emotional state based on their behavior and facial expressions.

[0684] This invention is an integrated system that optimizes efficiency and quality in the manufacturing process and enables responses based on the user's psychological state. The system comprises an information acquisition device, an analysis means including machine learning, a notification device, a suggestion device, an implementation device, and an emotion analysis device that analyzes the user's emotional state.

[0685] First, the terminal acquires data in real time via sensors and devices installed on the manufacturing line and transmits it to a server. This data includes temperature, humidity, pressure, and operating status.

[0686] The server analyzes data acquired using a generative AI model and applies machine learning techniques to detect anomalies. This allows for the rapid identification of anomalies, ensuring that the manufacturing process is maintained optimally.

[0687] When an anomaly is detected, the server immediately notifies the user through an alerting device. In addition, an emotion analysis device analyzes the user's emotional state in real time, and uses the analysis results to adapt and adjust the tone of the notification and response measures. This allows for a response that minimizes the burden on the user, even when they are feeling stressed or anxious.

[0688] For example, if the temperature shows an abnormal value during the manufacturing process, the notification will include a prompt to check the cooling system, along with a message such as, "Please remain calm. We will provide you with detailed instructions."

[0689] A concrete example of a prompt message would be: "A data anomaly has been detected on the production line. The user may be feeling anxious. What reassuring message should be sent?"

[0690] In this way, the server and terminal work together to create a system that enables the smooth operation of the manufacturing process while taking into account the user's emotional state.

[0691] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0692] Step 1:

[0693] The terminal collects data from various sensors placed on the manufacturing line. Specifically, temperature, humidity, pressure, and machine operating status are acquired in real time. This input data is collected as basic information for monitoring the manufacturing process.

[0694] Step 2:

[0695] The terminal sends the collected data to the server. The data is transmitted via a communication protocol and delivered to the server in real time over the network. Through this process, the server obtains the data for analysis.

[0696] Step 3:

[0697] The server analyzes the received data using a generating AI model. Specifically, machine learning techniques are applied to detect anomaly patterns in the data. It analyzes real-time information as input data, and if an anomaly is detected, it outputs the result as an anomaly summary.

[0698] Step 4:

[0699] The server immediately notifies the user via an alerting device if an anomaly is detected. The notification includes an explanation of the anomaly, specific countermeasures, and encouraging or cautionary messages generated by a generative AI model. In this step, the notification content is adjusted based on the results of sentiment analysis, aiming to reduce the user's psychological burden.

[0700] Step 5:

[0701] Upon receiving a notification, the user receives guidance from the sentiment analysis device and makes appropriate adjustments. Based on the feedback from the sentiment analysis device, the server provides the next optimal prompts and suggestions to support the user. The user's behavior and responses are continuously monitored, and the sentiment data is used for further analysis.

[0702] This series of steps creates a system where servers and terminals work together to provide users with optimal guidance for the manufacturing process.

[0703] (Application Example 2)

[0704] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0705] Traditional manufacturing process management lacks consideration for the emotional state of users, leading to problems such as decreased work efficiency and quality. Furthermore, the lack of information provided to enable appropriate responses when anomalies are detected can cause users to experience excessive stress.

[0706] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0707] In this invention, the server includes means for acquiring information to perform real-time observation of the manufacturing process, means for analyzing the acquired information including machine learning, and means for integrating an emotion analysis engine to analyze the user's emotional state and provide optimal support according to the user's emotions. This enables efficient and high-quality management of the manufacturing process while taking the user's emotional state into consideration.

[0708] "Information acquisition means for real-time observation of the manufacturing process" refers to means for collecting necessary data in real time during manufacturing operations and processing it immediately.

[0709] "Analysis methods including machine learning to analyze acquired information" refers to methods that incorporate machine learning techniques to detect anomalies or patterns hidden within collected data.

[0710] "A means of integrating an emotion analysis engine that analyzes the user's emotional state and providing optimal support according to the user's emotions" refers to a means of evaluating the user's emotional state by analyzing their facial expressions and operation patterns, and then providing support accordingly.

[0711] The system for implementing this invention can acquire and analyze information during the manufacturing process and provide appropriate support tailored to the user's emotional state. The server uses information acquisition means to collect data from the manufacturing process in real time. This information is expected to be provided from terminals equipped with sensors and cameras. The acquired data is then analyzed by analysis means including machine learning. In this process, machine learning algorithms are used to detect known anomalies and patterns that can be improved.

[0712] Furthermore, an emotion analysis engine is integrated to analyze the user's emotional state. This engine analyzes behavioral data such as the user's facial expressions, operation speed, and error frequency, and uses this to infer the user's emotional state. The results of this analysis are used to provide the user with the best possible support. For example, if the user is feeling stressed, the notification tone may be softened and encouraging messages added to reduce the user's burden.

[0713] This system uses the OpenCV library and Keras to perform facial recognition and emotion analysis. Based on the analysis results, an SDK can be used for robot control to provide appropriate communication. For example, if a worker on a manufacturing line shows signs of fatigue, the emotion analysis engine can detect that emotion, and the robot can deliver a message such as, "You've worked hard. Why don't you take a short break?" An example of a prompt message to input into the AI ​​model generated based on this invention is, "Consider a factory robot system that analyzes an employee's facial expression to determine their emotion and provides optimal support if they are stressed."

[0714] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0715] Step 1:

[0716] The server receives data in real time from terminals equipped with sensors and cameras on the manufacturing line. This input data includes information about the progress of manufacturing and the facial expressions of workers. This data is first stored in a database and used as foundational data for subsequent analysis.

[0717] Step 2:

[0718] The server analyzes the acquired data using machine learning algorithms to detect signs of anomalies. This analysis includes data processing and classification to identify abnormal operating patterns and product quality degradation. The output generates detailed information about the detected anomalies.

[0719] Step 3:

[0720] The server uses an emotion analysis engine to analyze the user's emotional state. Input data includes the user's operation speed, error frequency, and facial expression data. Through data processing, this data is output as the user's stress and fatigue levels.

[0721] Step 4:

[0722] The server generates optimal support for the user based on anomaly notifications and analysis of emotional state. Specifically, it generates notifications in a gentle tone and sends them with encouraging messages. This process uses a generative AI model, which generates different messages depending on the prompt.

[0723] Step 5:

[0724] After the user receives a notification, the server monitors whether adjustments have been made in response to the notification and provides further support as needed. Throughout the entire process, the terminal and server exchange information in real time to optimize system operation.

[0725] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0726] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0727] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0728] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0729] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0730] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0731] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0732] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0733] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0734] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0735] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0736] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0737] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0738] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0739] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0740] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0741] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0742] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0743] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0744] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0745] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0746] The following is further disclosed regarding the embodiments described above.

[0747] (Claim 1)

[0748] A data acquisition method for real-time monitoring of the manufacturing process,

[0749] Analysis means including artificial intelligence for analyzing acquired data,

[0750] A notification means that detects and notifies of anomalies based on the analysis results,

[0751] A suggestion method for recommending adjustments to the notified anomaly,

[0752] An execution mechanism that puts the adjusted conditions into action and monitors them again,

[0753] A system that includes this.

[0754] (Claim 2)

[0755] The system according to claim 1, wherein the adjustments recommended by the proposed means are aimed at optimizing the manufacturing process.

[0756] (Claim 3)

[0757] The system according to claim 1, wherein the notification means immediately notifies the user when an abnormality is detected.

[0758] "Example 1"

[0759] (Claim 1)

[0760] A device that collects environmental information using measuring devices placed in the manufacturing process and transmits it via communication means,

[0761] An analysis means having a device that preprocesses received information and analyzes it using a machine learning model,

[0762] A perceptual means that presents anomalies detected from the analysis results using an output device, prompting human response,

[0763] Proposal means including a proposal device that proposes a corrective procedure based on the presented anomaly,

[0764] An execution means having the function of executing the proposed modification, storing the modified state for re-evaluation, and providing feedback,

[0765] A system that includes this.

[0766] (Claim 2)

[0767] The system according to claim 1, wherein the proposed modification is intended to improve the efficiency of the manufacturing process.

[0768] (Claim 3)

[0769] The system according to claim 1, wherein information is immediately transmitted to a human when an abnormality is detected by a perceptual means.

[0770] "Application Example 1"

[0771] (Claim 1)

[0772] A means of acquiring information for real-time monitoring of the manufacturing process,

[0773] An analysis means including a machine learning algorithm for analyzing the acquired information,

[0774] A means of communication that detects anomalies based on analysis results and immediately notifies the administrator,

[0775] A suggestion mechanism that recommends adjustments to the notified anomaly and provides a suggested operation,

[0776] Implementation means for performing the adjusted work and monitoring it again,

[0777] A user interface means for visually displaying anomaly detection information,

[0778] A system that includes this.

[0779] (Claim 2)

[0780] The system according to claim 1, wherein the adjustments recommended by the proposed means are intended to improve the efficiency of the production process.

[0781] (Claim 3)

[0782] The system according to claim 1, which immediately notifies an operator when an abnormality is detected in the communication means.

[0783] "Example 2 of combining an emotion engine"

[0784] (Claim 1)

[0785] Information acquisition device means for monitoring the manufacturing process in real time,

[0786] Analysis methods, including machine learning, for analyzing acquired information,

[0787] A notification device means that detects an anomaly based on the analysis results and notifies the user of the anomaly,

[0788] A suggestion device means that proposes adjustments for the notified abnormality,

[0789] An implementation device means that puts the adjusted conditions into action and monitors them again,

[0790] A means of using an emotion analysis device to analyze the emotional state of a user,

[0791] A system that includes this.

[0792] (Claim 2)

[0793] The system according to claim 1, wherein the adjustments recommended by the proposed device are aimed at optimizing the manufacturing process.

[0794] (Claim 3)

[0795] The system according to claim 1, which immediately notifies the user when an abnormality is detected by the notification device and adjusts the content of the notification based on the results of the emotion analysis device.

[0796] "Application example 2 when combining with an emotional engine"

[0797] (Claim 1)

[0798] Information acquisition means for real-time observation of the manufacturing process,

[0799] Analytical means including machine learning to analyze acquired information,

[0800] A notification means that detects and notifies of anomalies based on the analysis results,

[0801] An advisory tool that recommends adjustments to the notified anomaly,

[0802] An execution means for putting the adjusted state into action and observing it again,

[0803] This system integrates an emotion analysis engine to analyze the user's emotional state and provides a means to offer optimal support tailored to the user's emotions.

[0804] A system that includes this.

[0805] (Claim 2)

[0806] The system according to claim 1, wherein the adjustments recommended by the advisory means are intended to improve the efficiency of the manufacturing process.

[0807] (Claim 3)

[0808] The system according to claim 1, wherein the notification means immediately notifies the operator when an abnormality is detected. [Explanation of Symbols]

[0809] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of acquiring information for real-time monitoring of the manufacturing process, An analysis means including a machine learning algorithm for analyzing the acquired information, A means of communication that detects anomalies based on analysis results and immediately notifies the administrator, A suggestion mechanism that recommends adjustments to the notified anomaly and provides a suggested operation, Implementation means for performing the adjusted work and monitoring it again, A user interface means for visually displaying anomaly detection information, A system that includes this.

2. The system according to claim 1, wherein the adjustments recommended by the proposed means are intended to improve the efficiency of the production process.

3. The system according to claim 1, which immediately notifies an operator when an abnormality is detected in the communication means.

Images