System

Abstract

A system is provided.SOLUTION: A system comprising: means for converting test data into an electronic format and storing the test data; means for cleansing the test data and removing incomplete data; means for standardizing the cleansed test data; means for storing the standardized data in a recording medium; means for acquiring and recording progress status at fixed time intervals; means for displaying the acquired progress status in real time; and means for updating and managing the recorded progress status information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the medical field, managing large volumes of test data and understanding progress are major challenges. In particular, analyzing and reporting test results requires a lot of manpower and time, so efficiency is required. In addition, real-time management of test progress, data visualization, and accurate and rapid disease prediction are essential for medical professionals to make quick decisions. Against this background, there is a need for systems that can reduce the burden on medical professionals and enable rapid response to patients. [Means for solving the problem]

[0005] This invention is a system that automatically converts test data into an electronic format, cleansing it, removing incomplete data, and standardizing it, and provides a means for acquiring progress at regular intervals and displaying it in real time. The system also includes a means for visualizing the test data, displaying it as a graph, and saving it as an image file. It also includes a means for predicting a disease name from test results using a trained prediction model and recording the prediction result by adding it to the test data. This significantly improves the efficiency of test data management, reduces the burden on medical professionals, and enables quick and accurate reporting of test results.

[0006] "Test Data" refers to information related to a patient's health status collected by a medical institution or testing institution.

[0007] "Electronic format" refers to the form or protocol for storing and processing data electronically.

[0008] "Cleansing" refers to the process of removing or correcting incomplete data to ensure consistency and accuracy.

[0009] "Incomplete data" refers to data that contains missing values ​​or outliers and is not suitable for analysis as is.

[0010] "Standardization" refers to the process of transforming data to standardize variability, usually by adjusting the data relative to a mean value.

[0011] "Status" refers to information that indicates the current state or stage of an inspection or work.

[0012] "Real-time display" refers to the function of providing data and information to users in a visualized form immediately as soon as it is acquired.

[0013] "Visualization" refers to converting data into a visual format, such as a graph or chart, that makes the data easier to understand.

[0014] A "predictive model" refers to a computational algorithm or statistical model that uses historical data to predict future outcomes or trends.

[0015] "Disease prediction" refers to the process of determining the likelihood of a specific disease or condition based on test data.

[0016] "Storage means" refers to a method or device for storing data so that it can be reused or referenced at a later time. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system analyzes and manages test data uploaded by users through various processes, improving the work efficiency of medical professionals.

[0039] System Configuration

[0040] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[0041] Automatic processing of inspection data

[0042] Uploading data

[0043] The user uploads a CSV file of the test data to the server through the system interface, and the server saves the file in a specified directory.

[0044] Data cleansing and standardization

[0045] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[0046] Automated progress management

[0047] Regular progress capture and recording

[0048] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[0049] Data Visualization and Analysis

[0050] Data Visualization

[0051] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and provided to the user for viewing via their device.

[0052] Prediction of likely test results

[0053] Use of disease prediction models

[0054] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[0055] Specific examples

[0056] Example 1: A concrete example of automated processing of test data

[0057] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[0058] Example 2: Specific example of automated progress management

[0059] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[0060] Example 3: Specific examples of data visualization and analysis

[0061] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[0062] Example 4: Specific example of predicting the likelihood of test results

[0063] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[0064] These features will significantly improve the work efficiency of medical professionals, enabling them to respond quickly and accurately, and are expected to enhance the quality of service provided to patients, leading to improvements in the overall medical process.

[0065] The processing flow will be explained below.

[0066] Automatic processing of inspection data

[0067] Step 1:

[0068] The user uploads the CSV file of the test data to the server through the system interface. The uploaded file is saved in a specified directory on the server.

[0069] Step 2:

[0070] The server reads the uploaded CSV file. Once the reading is complete, it starts cleansing the data, identifying incomplete data (e.g. rows with missing values ​​or outliers) and removing these rows.

[0071] Step 3:

[0072] The server standardizes the cleansed data by calculating the mean and standard deviation for each data column and standardizing each value (subtracting the mean and dividing by the standard deviation).

[0073] Step 4:

[0074] The standardized data is saved as a new CSV file on the storage device. After the clean data has been saved, the save path is recorded in the log.

[0075] Automated progress management

[0076] Step 1:

[0077] The server obtains the current inspection progress status at regular intervals (for example, every 60 seconds), which includes information such as the inspection progress status and the degree of completion of processing.

[0078] Step 2:

[0079] The server writes the obtained progress status to a file in text format. Specifically, it appends information combining the current timestamp and progress status to the progress_status.txt file.

[0080] Data Visualization and Analysis

[0081] Step 1:

[0082] The server loads the previously saved clean data file (e.g., cleaned_exam_data.csv) and prepares the data for visualization.

[0083] Step 2:

[0084] The server visualizes the data, specifically generating time series graphs and adding appropriate labels and titles. A dedicated library is used to generate the graphs.

[0085] Step 3:

[0086] Save the generated graph as an image file, for example, as exam_data_visualization.png, and record the save path in the log.

[0087] Prediction of likely test results

[0088] Step 1:

[0089] The server loads a pre-trained predictive model (e.g., disease_prediction_model.joblib) that is used to predict the likelihood of disease based on laboratory data.

[0090] Step 2:

[0091] The server inputs the clean data into the predictive model and performs disease prediction: it feeds each row of data into the model and gets the predicted disease outcome.

[0092] Step 3:

[0093] The prediction results are appended to the test data. The appended data is saved as a new CSV file (e.g., exam_data_with_predictions.csv), and the save path is recorded in the log.

[0094] Step 4:

[0095] The user views the prediction results through a terminal, and uses the system interface to display the saved data with prediction results and use them as a reference for diagnosis.

[0096] Through these steps, the system efficiently realizes automatic processing of test data, automated progress management, data visualization and analysis, and probable prediction of test results.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] In the medical field, management and analysis of test data is often done manually, resulting in reduced efficiency and potential for errors. It is also difficult to grasp progress, which can lead to delayed diagnoses. Furthermore, while there is a need for automated visualization of test data and disease prediction, there is a problem that no system exists that comprehensively solves these issues.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes: a means for a user to upload test data; a means for saving the test data in a specified directory; a means for cleansing the test data to remove incomplete data; a means for standardizing the cleansed test data; a means for saving the standardized data on a recording medium; a means for acquiring and recording progress status at regular time intervals; a means for displaying the acquired progress status in real time; a means for updating and managing the recorded progress status information; a means for generating a time-series graph based on the cleansed and standardized test data; a means for saving the graph as an image file; a means for predicting a disease name from the test results using a pre-trained prediction model; and a means for adding and recording the prediction results to the test data. This automates the management and analysis of test data, improves work efficiency, and reduces errors. Furthermore, real-time progress monitoring is possible, resulting in faster and more accurate diagnoses.

[0102] "Test data" means electronic records of information containing the results of medical tests.

[0103] The "means for users to upload test data" refers to an interface that allows users to send test data to the system via their terminals.

[0104] The "means for saving test data in a specified directory" is a function for saving test data received by the server in a specific folder.

[0105] The "means for cleansing test data and removing incomplete data" refers to a process for analyzing uploaded test data and removing missing or erroneous data.

[0106] "Means to standardize cleansed laboratory data" refers to the process of converting data to a unified scale to ensure consistency when inputting data into analytical and predictive models.

[0107] The "means for storing standardized data on a recording medium" is a function for storing test data in a digital format after standardization has been completed.

[0108] The "means for acquiring and recording the progress status at regular intervals" is a system function in which the server periodically monitors the progress of the inspection work and records the information.

[0109] The "means for displaying the acquired progress status in real time" is an interface that displays the recorded progress information so that the user can monitor it in real time.

[0110] The "means for updating and managing recorded progress status information" is a system function that appropriately updates progress information and corrects or supplements it as necessary.

[0111] The "means for generating a time series graph based on cleansed and standardized test data" is a function for creating a graph that visually represents data fluctuations based on processed test data.

[0112] The "means for saving the graph as an image file" is a function for saving the generated time series graph in image format.

[0113] "Means for predicting disease names from test results using a pre-trained predictive model" is a function that predicts diseases from new test data using a model trained on past data.

[0114] The "means for adding and recording prediction results to test data" is a function for adding the predicted disease name to existing test data and resaving it.

[0115] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system aims to improve the work efficiency of medical professionals by analyzing and managing test data uploaded by users on a server.

[0116] System Configuration

[0117] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[0118] Hardware and software used

[0119] The server is a computer system equipped with a high-performance processor and sufficient memory, running a Linux (registered trademark)-based OS and Apache (registered trademark) HTTP Server. Data is stored in an SQL-based database (e.g., MySQL (registered trademark)).

[0120] The terminal is a PC or tablet operated by the user, and the user accesses the system through a web browser such as Chrome or Firefox. The web interface is developed using HTML, CSS, and JavaScript (registered trademark).

[0121] Explaining program processing in natural language

[0122] 1. User uploads inspection data

[0123] Through the system interface, users upload test data CSV files to the server, which then stores the files in a specified directory.

[0124] 2. The server performs data cleansing and standardization

[0125] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[0126] 3. The server periodically acquires and records the progress

[0127] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[0128] 4. The server visualizes the data

[0129] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and made available to users via their devices.

[0130] 5. The server performs disease prediction

[0131] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[0132] Specific examples

[0133] Example 1: A concrete example of automated processing of test data

[0134] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[0135] Example 2: Specific example of automated progress management

[0136] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[0137] Example 3: A concrete example of data visualization and analysis

[0138] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[0139] Example 4: Specific example of predicting the likelihood of test results

[0140] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[0141] Prompt Sentence Examples

[0142] Upload the following test data for cleansing, standardization, and disease prediction.

[0143] Filename: exam_data.csv

[0144] This prompt causes the system to automatically begin processing the test data and generate results.

[0145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0146] System program processing flow

[0147] Processing Steps

[0148] Step 1:

[0149] The user uploads the test data.

[0150] The user opens a web browser on their device and accesses the system interface (HTML form). Using the file selection dialog on the interface, they select the CSV file of the examination data (e.g., exam_data.csv) and click the "Upload" button. The server receives this file and saves it in a specified directory (e.g., / data / uploads / ).

[0151] Input: exam_data.csv (file uploaded by user)

[0152] Output: exam_data.csv saved in the specified directory

[0153] Step 2:

[0154] The server performs data cleansing.

[0155] The server reads the CSV files stored in the specified directory, analyzes the data, and removes rows containing incomplete data or missing values. The cleansed data is temporarily stored in temporary variables in memory.

[0156] Input: exam_data.csv in the specified directory

[0157] Output: Cleansed data (in memory) with imperfect data removed

[0158] Step 3:

[0159] The server performs the normalization of the data.

[0160] The server calculates the mean and standard deviation for each numeric item based on the parsed data, then normalizes each data point using these statistics. The normalized data is saved as a new CSV file (e.g., cleaned_exam_data.csv).

[0161] Input: Cleansed data (in memory)

[0162] Output: cleaned_exam_data.csv containing the standardized data

[0163] Step 4:

[0164] The server periodically obtains and records the progress.

[0165] The server checks the inspection progress every 60 seconds and records the progress data in a text file (e.g., progress_status.txt). This information includes the current data processing status and whether or not there were any errors.

[0166] Input: Internal server progress data

[0167] Output: progress_status.txt (latest progress information)

[0168] Step 5:

[0169] The server visualizes the data.

[0170] The server generates a time series graph based on the standardized data. This graph is saved in image format (e.g., PNG format). The graph is generated using the Python matplotlib library.

[0171] Input: cleaned_exam_data.csv (standardized data)

[0172] Output: exam_data_visualization.png (generated graph)

[0173] Step 6:

[0174] The server performs the disease prediction.

[0175] The server loads a pre-trained machine learning model (e.g., Scikit-learn's random forest model). Standardized test data is input into this model to predict the likelihood of disease. The prediction results are appended to the original data and saved as a new CSV file (e.g., exam_data_with_predictions.csv).

[0176] Input: Standardized test data, trained prediction model

[0177] Output: exam_data_with_predictions.csv (data with prediction results)

[0178] Step 7:

[0179] The user views the results.

[0180] Users can access the system interface through their terminal and download or view the prediction result file (e.g., exam_data_with_predictions.csv), allowing them to check the prediction results and use them as a reference for diagnosis.

[0181] Input: exam_data_with_predictions.csv (data with prediction results)

[0182] Output: Prediction result data viewed by the user

[0183] (Application example 1)

[0184] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0185] In modern factories, progress management and anomaly detection in production processes are important issues. However, there are insufficient means to efficiently carry out these management and detection tasks, which requires a lot of time and effort. In addition, it is difficult to quickly notify and respond when an anomaly occurs, which often leads to a decline in productivity and quality. Therefore, there is a need for efficient real-time progress management, anomaly detection, and data visualization.

[0186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0187] In this invention, the server includes means for converting the inspection data into an electronic format and saving it, means for cleansing the inspection data to remove incomplete data, means for standardizing the cleansed inspection data, means for saving the standardized data on a recording medium, means for acquiring and recording a progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for detecting the possibility of an anomaly occurring using an anomaly detection model, means for notifying a detected anomaly via a smartphone, and means for displaying the data as a graph and saving it as an image file. This enables efficient management of the progress status of the production process, enables rapid response when an anomaly occurs, and is expected to improve productivity and quality.

[0188] "Inspection data" is information that includes measurements and status information related to a factory or manufacturing process.

[0189] "Cleansing" is the process of removing missing or outliers from data and shaping the data.

[0190] "Standardization" is the process of reducing variability in data and converting it to a uniform scale.

[0191] A "recording medium" is a physical or electronic means for storing data.

[0192] "Progress" is information that indicates the progress of work in a factory or manufacturing process.

[0193] "Real-time" means that the latest information is always acquired and displayed in real time.

[0194] "Update management" is the management process for keeping information and data up to date.

[0195] An "anomaly detection model" is a machine learning model for detecting abnormal values ​​in data.

[0196] "Anomaly" refers to a condition or occurrence that is different from the norm, including problems in the manufacturing process.

[0197] A "smartphone" is a mobile device that can connect to the Internet and use multifunctional applications.

[0198] "Graphical display of data" is a method of converting and displaying data in a form that is visually easy to understand.

[0199] An "image file" is a digital file format for storing visual information.

[0200] A "trained predictive model" is a model that has been trained using past data and can predict future states.

[0201] "Progress delay" is a phenomenon that indicates a delay between planned progress and actual progress.

[0202] "Production progress" is information indicating the current progress of work in the manufacturing process.

[0203] The system for realizing this invention is mainly composed of three elements: a server, a terminal, and a user. Each specific processing step will now be described.

[0204] 1. Upload your data

[0205] Users upload data generated by factory robots and manufacturing machines to a server in CSV file format using a smartphone interface.

[0206] 2. Data Cleansing and Standardization

[0207] The server receives the uploaded data, removes incomplete data, reduces the data variability, and converts it to a standard scale.

[0208] 3. Data storage

[0209] The cleansed and standardized data is stored on a storage medium, which is typically a database (e.g., MySQL or PostgreSQL).

[0210] 4. Automated progress management

[0211] The server collects and records the factory progress at regular intervals, and this information is updated in real time and can be viewed by users via a smartphone app.

[0212] 5. Anomaly detection and notification

[0213] The server analyzes the data using a pre-trained anomaly detection model (using Tensorflow (registered trademark) or PyTorch) to detect anomalies. Detected anomalies are notified to the user via smartphone.

[0214] 6. Data visualization and analysis

[0215] The server generates a time series graph based on the cleansed and standardized data, which is saved as an image file (PNG file) and displayed to the user via a smartphone app.

[0216] 7. Progress forecast using predictive models

[0217] The server uses the trained prediction model to predict possible delays in progress or abnormalities. The prediction results are added to the data and saved again. This allows users to visually check the actual progress and take any necessary measures quickly.

[0218] Specific examples

[0219] For example, if a production line in a factory is behind schedule or if there is a possibility of a machine malfunction, the system will automatically visualize the progress and potential malfunction and warn the user, who can then check the progress in real time via a smartphone app.

[0220] Prompt Sentence Examples

[0221] Examples of prompts for generative AI models include:

[0222] Analyze factory production line data and predict the possibility of anomalies. Visualize progress taking into account production speed and machine status, and generate alerts if delays or anomalies are detected.

[0223] This will enable efficient management of factory production processes and improve productivity and quality.

[0224] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0225] Step 1:

[0226] The user uploads data generated by factory robots and manufacturing machines to the server in CSV file format. Specifically, the user uses a smartphone app to select the desired CSV file from the file selection screen. This input data includes numerical values ​​and status information about the manufacturing process. The server saves the received file in a temporary directory.

[0227] Step 2:

[0228] The server reads the uploaded CSV file and cleanses the data, specifically removing incomplete data and converting it to the required format. The input at this stage is the CSV file and the output is a cleansed DataFrame. The software used is Python and the Pandas library.

[0229] Step 3:

[0230] The server standardizes the cleansed data. Specifically, it calculates the mean and standard deviation of each data and standardizes the data. This standardized data is stored in a database. The input is a cleansed data frame, and the output is a standardized data frame. The software used is Python and the Scikit-learn library.

[0231] Step 4:

[0232] The server obtains the progress status at regular intervals and records that information. Specifically, the server saves the progress data it obtains from each manufacturing machine by appending it to a text file. The input is the progress data, and the output is a text file containing the progress status. This process is performed periodically using scheduling software.

[0233] Step 5:

[0234] The server displays the acquired progress status in real time. Specifically, it updates and displays the progress status information in real time on a web interface or smartphone app. The input is the progress status data, and the output is the latest progress status displayed on the user interface.

[0235] Step 6:

[0236] The server uses an anomaly detection model to detect possible anomalies in the data. Specifically, it inputs the cleansed and standardized data into the AI ​​model and calculates an anomaly score. The input is the standardized data, and the output is the anomaly score. The software used is TensorFlow or PyTorch.

[0237] Step 7:

[0238] The server notifies the user of detected anomalies via their smartphone. Specifically, if the anomaly score exceeds a threshold, it sends a push notification to the smartphone app. The input is the anomaly score, and the output is a notification message to the user.

[0239] Step 8:

[0240] The server generates a time series graph based on the cleansed and standardized data. Specifically, it generates the graph using Matplotlib or Plotly and saves it as an image file. The input is the standardized data, and the output is a time series graph image.

[0241] Step 9:

[0242] The server uses a trained prediction model to predict the possibility of progress delays or abnormalities. Specifically, it inputs standardized data into the prediction model and obtains a prediction result. The input is the standardized data, and the output is the prediction result. The software used is TensorFlow and PyTorch.

[0243] Step 10:

[0244] The server adds the prediction results to the data and saves them back to the database. Specifically, the obtained prediction results are added to the existing data frame and saved as a new one. The input is the prediction results and the existing data frame, and the output is the updated data frame.

[0245] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0246] This invention is a system that combines automatic processing of medical test data, automated progress management, data visualization and analysis, and probable test result prediction in the medical field with an emotion engine that recognizes the user's emotions. This system uses the emotion engine to monitor the user's emotions and provides appropriate interfaces and feedback according to the user's emotional state, thereby improving the work efficiency of medical professionals and the patient experience.

[0247] System Configuration

[0248] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running predictive models, and operating the emotion engine, while the terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[0249] Introducing the Emotion Engine

[0250] User Emotion Recognition

[0251] The server runs an emotion engine based on the user's voice, facial expressions, text input, etc. to recognize the user's emotional state, and provides appropriate feedback according to the user's emotional state.

[0252] Dynamically changing the display of test data based on emotions

[0253] The server dynamically changes how the test data is displayed based on the user's perceived emotions, for example, by displaying more detailed information and explanations if the user is anxious, to provide a sense of security.

[0254] Emotional state monitoring and collaboration

[0255] The emotion engine continuously monitors the user's emotional state, and the results are recorded in relation to progress, allowing changes in the user's mental state to be tracked.

[0256] Specific examples

[0257] Example 1: Emotion Recognition

[0258] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Based on this information, the server displays an interface that matches the user's mood.

[0259] Example 2: Dynamic data display

[0260] When the user tries to check the test results, the server recognizes that the user is anxious using its emotion engine. In addition to the usual display method, the server displays detailed explanations of the results and reference information to alleviate the user's anxiety.

[0261] Example 3: A concrete example of linking and monitoring emotional states

[0262] The server simultaneously runs an emotion engine to monitor the user's emotional state while the user checks the test data and views the progress. The results are recorded along with the progress so that medical professionals can later understand the user's emotional changes.

[0263] This system not only improves the work efficiency of medical professionals, but also provides services that take into consideration the mental health of patients. By combining it with an emotion engine, it becomes possible to provide responses that are optimized for each individual user, resulting in the provision of higher quality medical services.

[0264] The processing flow will be explained below.

[0265] Embodiment of a system combining emotion engines

[0266] Recognizing user emotions with an emotion engine

[0267] Step 1:

[0268] The user logs into the system.

[0269] The server receives the user's login information and performs authentication.

[0270] Step 2:

[0271] When the user initiates an action, the device's camera and microphone become active.

[0272] The device collects the user's facial expressions and voice data in real time.

[0273] Step 3:

[0274] The server sends the collected facial expression and voice data to the emotion engine.

[0275] The emotion engine uses facial expression analysis and voice analysis algorithms to determine the user's emotional state.

[0276] Step 4:

[0277] The emotion engine determines the user's emotion and returns the result to the server.

[0278] The server receives the data and records the user's current emotional state.

[0279] Dynamically changing the display of test data based on emotions

[0280] Step 1:

[0281] The user performs operations to view the inspection data.

[0282] The terminal sends the request to the server.

[0283] Step 2:

[0284] The server ascertains the user's emotional state.

[0285] Refer to the data of the emotional state determined by the emotion engine.

[0286] Step 3:

[0287] The server changes how the test data is displayed based on the user's emotional state.

[0288] For example, if the user is in an anxious state, detailed explanations or reassuring messages are displayed.

[0289] Step 4:

[0290] The display content is transmitted to the terminal based on the changed display method.

[0291] The terminal displays the test data in a user-readable format.

[0292] Emotional state monitoring and collaboration

[0293] Step 1:

[0294] While the user is using the system, the device continuously collects facial expression and voice data.

[0295] The collected data is periodically sent to a server.

[0296] Step 2:

[0297] The server periodically sends data to the emotion engine to update the user's emotional state.

[0298] The updated emotion data is saved to the server in real time.

[0299] Step 3:

[0300] The server integrates the progress status data and the emotion data, and records the changes in the user's emotional state in association with the progress status.

[0301] The recorded data is later used by medical personnel to ascertain the user's emotional state.

[0302] Step 4:

[0303] When the user exits the system, the server records the final emotional state and ends the session.

[0304] The device will disable the camera and microphone and stop transmitting collected data.

[0305] <<Example>>

[0306] Example 1: User Emotion Recognition

[0307] When a user logs in to the system, the device's camera and microphone collect facial expressions and voice in real time. The server sends this data to an emotion engine to recognize when the user is feeling stressed. Based on the results, the server displays an interface using a theme color that has a relaxing effect on the user.

[0308] Example 2: Dynamic data display based on emotions

[0309] When a user attempts to view a particular test result, the server detects through an emotion engine that the user is feeling anxious, and the device displays the test data with detailed explanations and a message saying, "If you have any questions, please consult your doctor."

[0310] Example 3: Monitoring and recording emotional states

[0311] While the user is checking their progress, the server uses an emotion engine to monitor their emotional state and records the results along with their progress. If the user becomes frustrated, the server notifies medical professionals and prompts them to take appropriate action.

[0312] These steps will enable the system to achieve more flexible and efficient medical data management and provision that takes into account users' emotions.

[0313] Example 2

[0314] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] The modern medical field requires efficient processing and management of test data. However, existing systems are unable to adequately address the following: test data cleansing, standardization, storage, real-time display and recording of progress, data visualization, and disease prediction using predictive models. They also lack the ability to recognize the user's emotional state and provide appropriate feedback and interfaces accordingly. This makes it difficult to improve the work efficiency of medical professionals and the psychological satisfaction of patients.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for analyzing the user's voice, facial expression, and text input to recognize the user's emotional state, means for dynamically changing the display method of the test data based on the recognized emotional state, and means for continuously monitoring the user's emotional state and recording the results in association with the progress status. This enables efficient processing and management of test data and further enables the provision of appropriate feedback and interfaces according to the user's emotional state. This improves the work efficiency of medical professionals and increases the psychological satisfaction of patients.

[0317] "Test data" refers to information including results and measurements obtained by a user at a medical institution or testing facility.

[0318] "Cleansing" is the process of removing incomplete data and errors from test data and organizing it.

[0319] "Standardization" is the process of aligning cleansed data into a consistent format and units.

[0320] A "recording medium" is a digital storage device or service for storing data.

[0321] "Progress" is information that indicates the progress and current status of medical treatments and examinations.

[0322] "Emotional state" refers to the user's psychological state or mood.

[0323] "Voice analysis" is a technology that analyzes voice data and extracts specific patterns and emotions.

[0324] "Facial expression analysis" is a technology that analyzes facial expressions captured by a camera and recognizes emotions.

[0325] "Text analysis" is the process of analyzing textual information entered by a user to understand their emotions and intent.

[0326] "Dynamic change" is a mechanism that changes the display content and behavior of the system in real time according to specific conditions or situations.

[0327] "Monitoring" is the act of continuously observing a user's state and behavior and collecting data.

[0328] "Feedback" is information or a response provided to a user with the purpose of approving or correcting their behavior.

[0329] This system is specialized for efficient processing and management of medical test data. It automates a series of processes, from test data cleansing to standardization, storage, and sentiment analysis, with the aim of improving the work efficiency of medical professionals and the psychological satisfaction of patients.

[0330] System Configuration

[0331] The system consists of three components: a server, a terminal, and a user. The server processes and stores data, runs predictive models, and operates the emotion engine. The terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[0332] Hardware and software used

[0333] Hardware: Use a server equipped with a high-performance processor and a large amount of memory (e.g., Dell PowerEdge, HP ProLiant) as the server. Use a personal computer or tablet equipped with a camera and microphone as the terminal.

[0334] Software: The server uses "EmotionAPI," "database management system (e.g., MySQL, PostgreSQL)," and "web server (e.g., Apache, Nginx)." The terminal uses technologies such as "WebRTC" and "JavaScript framework (e.g., React.js)."

[0335] Data processing and calculation

[0336] 1. Cleansing: The server receives the uploaded test data and filters out missing or inappropriate data.

[0337] 2. Standardization: The cleansed data is converted into a unified format and stored in a database.

[0338] 3. Emotion recognition: The user's voice, facial expressions, and text input sent from the device are analyzed using the "Emotion API" to recognize the user's emotional state.

[0339] 4. Dynamic Change: The server dynamically changes how the test data is displayed based on the perceived emotional state. For example, if the user is anxious, it provides more information and reassuring feedback.

[0340] 5. Monitoring: The server continuously monitors the user's emotional state and records it in relation to their progress.

[0341] Specific examples

[0342] Example 1: Emotion Recognition

[0343] When a user logs in to the system, the device uses a camera and microphone to capture the user's facial expressions and voice. The server analyzes the data using the "Emotion API" to recognize the user's emotional state. If the user is feeling anxious, the server will provide an interface that provides a sense of security.

[0344] Example 2: Dynamic data display

[0345] When a user attempts to check the results of their test data, the device sends a request to the server. The server recognizes that the user is anxious and sends the results to the device with a detailed explanation and reference information. The device then displays the results to the user.

[0346] Example 3: A concrete example of emotional state monitoring

[0347] The server uses an emotion engine to monitor the user's emotional state while they are reviewing the test data and viewing their progress, and records the results in association with their progress so that medical professionals can review them later.

[0348] Prompt Sentence Examples

[0349] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Then, after reviewing the test data, the server displays the results with detailed explanations and provides feedback to the user to give them a sense of security.

[0350] This allows the system to not only efficiently process and manage test data, but also to provide an interface and feedback that takes user emotions into consideration, thereby improving the work efficiency of medical professionals and increasing the psychological satisfaction of patients.

[0351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0352] Step 1:

[0353] When a user logs into the system, the terminal takes the user's email address and password as input and sends them to the server. The server queries a database to verify the authentication information. If authentication is successful, the server outputs a login success message to the terminal. The terminal receives this and notifies the user that the login was successful.

[0354] Step 2:

[0355] The device uses a camera and microphone to capture the user's facial expressions and voice. This captured data is sent as input to the server. The server analyzes the received data using the Emotion API to recognize the user's emotional state. Based on this result, the server generates appropriate user interface data and outputs it to the device. The device then displays the received interface data.

[0356] Step 3:

[0357] Users upload test data files using a drag-and-drop interface on their devices, which then send the files as input to the server, which cleanses the received test data and removes incomplete data. The cleansed data is then standardized and stored in a database.

[0358] Step 4:

[0359] The user requests confirmation of test data from the device. The device sends this request as input to the server. The server retrieves the relevant test data from the database and again uses the Emotion API to recognize the user's current emotional state. The server receives the test data and emotional state as input and generates detailed feedback. The device displays this feedback.

[0360] Step 5:

[0361] The device continuously captures the user's voice and facial expressions using a camera and microphone. This periodically captured data is sent as input to a server. The server uses the Emotion API to recognize the user's emotional state and record it in a database. The results are associated with progress and stored in the database. Medical professionals can access the database to check changes in the user's emotional state.

[0362] (Application example 2)

[0363] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0364] Conventional food delivery applications provide menu recommendations and feedback without considering the user's emotional state. As a result, it is difficult to provide appropriate support to users who are feeling stressed or anxious, limiting the quality of the user experience. Furthermore, they are unable to provide a dynamic interface based on the user's emotions, and improvements that would lead to increased user satisfaction are needed.

[0365] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for providing appropriate feedback based on the user's emotions using an emotion engine that recognizes the user's emotional state by analyzing the user's facial expressions and voice, and means for analyzing the user's facial expressions and presenting dietary recommendations based on the analysis. This enables feedback and recommendations that take the user's emotional state into consideration, significantly improving the quality of the user experience.

[0366] "Electronic test data format" refers to the technical means for reading test data, converting the data into a digital format, and storing it electronically.

[0367] "Cleansing" is the process of removing incomplete data and noise from inspection data to improve data quality.

[0368] "Standardization" is a technical means of converting cleansed data into a uniform format that makes it easier to compare and analyze.

[0369] "Storage medium" means a physical or electronic device or system for storing standardized data.

[0370] "Status capture" is the process of periodically reviewing and recording data and project progress.

[0371] "Real-time display" refers to a technical means that instantly reflects progress and data and displays them to the user.

[0372] Update management is the process of keeping recorded progress information up to date and managing changes and additions.

[0373] An "emotion engine" is an algorithm or technology that analyzes and recognizes a user's emotional state from their facial expressions and voice.

[0374] "Feedback" is a response that provides appropriate information or recommendations depending on the user's perceived emotional state.

[0375] "Dietary recommendations" are information that suggests meals and menus that are optimal for the user's emotional state at the time, based on the user's facial expression.

[0376] "Data visualization" is a technical method of displaying inspection data in the form of graphs and charts to make it easier to understand.

[0377] "Dynamic display change" is a technical means for changing the way data is displayed in real time according to the user's emotional state.

[0378] "Emotion monitoring" is the process of continuously tracking a user's emotional state and recording its changes.

[0379] A "predictive model" is an algorithm or technique that uses a pre-trained dataset to predict outcomes from new data.

[0380] A "prompt" is an instruction or command to be input into a generative AI model.

[0381] The present invention provides a system for recognizing a user's emotions in food delivery and providing appropriate feedback and recommendations based on the emotions. Specific embodiments of the system are described below.

[0382] System Configuration

[0383] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running the emotion engine, and operating the predictive model. The terminal provides an interface for users to access and operate the system. Users order food and check feedback.

[0384] Introducing the Emotion Engine

[0385] User Emotion Recognition

[0386] The server uses an image processing library (e.g., OpenCV) and an emotion recognition model (e.g., a model trained with Keras) to analyze the user's facial expressions. Images and videos uploaded by the user are taken, and the server processes them to recognize the user's emotional state.

[0387] Emotion-based dietary recommendations

[0388] The server provides appropriate feedback based on the recognized emotion, suggesting the most suitable menu from a predefined list to provide dietary recommendations according to the user's emotional state. For example, if the user is anxious, it will recommend relaxing meals and herbal teas.

[0389] Emotional state monitoring and collaboration

[0390] The server runs an emotion engine in real time to monitor the user's emotional state when confirming an order, records the results sequentially, and analyzes long-term trends to provide strategies to improve user satisfaction.

[0391] Specific examples

[0392] Examples of facial expression analysis and emotion recognition

[0393] When a user accesses the system, they take a picture of their face using the device's camera and upload it to the server. The server processes the image and recognizes the user's emotional state. Specific examples of prompts are as follows:

[0394] Analyze the image file "user_face.jpg" and recognize the user's emotions.

[0395] Provide appropriate feedback based on predicted emotions.

[0396] Dynamic Data Display and Feedback Example

[0397] When the user tries to confirm their order, the server analyzes the user's emotions and dynamically changes the feedback based on the results. For example, if the user is anxious, a recommendation message such as "Try a menu that has a relaxing effect" will be displayed.

[0398] Hardware and software used

[0399] Hardware: The user's device (such as a smartphone or tablet)

[0400] Software: Image processing libraries (e.g., OpenCV), machine learning frameworks (e.g., Keras), frameworks (e.g., Flask) on the server

[0401] In this way, personalized responses that take into account the user's emotional state can be made, significantly improving the user experience.

[0402] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0403] Step 1:

[0404] A user takes a picture of their face using the device camera and uploads it to the system. The input is the user's face image file, and the output is the image data sent to the server.

[0405] Step 2:

[0406] The server analyzes the received facial image data using the OpenCV library. Specifically, it uses a facial recognition algorithm to detect the face and extract the necessary parts. The input is the user's facial image data, and the output is image data of the facial area.

[0407] Step 3:

[0408] The server inputs the facial image data into an emotion recognition model trained with Keras to predict emotions. The input is the facial image data, and the output is the user's emotional state (e.g., "anxiety," "joy," "anger," etc.).

[0409] Step 4:

[0410] The server selects appropriate feedback and dietary recommendations based on the predicted emotional state. The input is the emotional state, and the output is a feedback message or recommended menu. For example, it performs a specific action such as "if the user is anxious, recommend a relaxing herbal tea."

[0411] Step 5:

[0412] The server sends the selected feedback and recommended menu to the terminal. The input is the feedback message and recommended menu, and the output is the information displayed on the terminal.

[0413] Step 6:

[0414] The terminal displays the feedback messages and recommended menus received from the server to the user. The input is the feedback information from the server, and the output is the information visually displayed to the user.

[0415] Step 7:

[0416] The server records the user's order history and feedback, and monitors changes in their emotional state. The input is the user's order and feedback data, and the output is continuous monitoring data. This allows us to analyze long-term user satisfaction and use it to further improve our services.

[0417] In this way, each processing step is performed sequentially, providing real-time feedback and recommendations to improve the user experience.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0419] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0421] [Second embodiment]

[0422] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0423] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0424] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0426] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0428] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0429] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0430] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0433] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0434] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system analyzes and manages test data uploaded by users through various processes, improving the work efficiency of medical professionals.

[0435] System Configuration

[0436] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[0437] Automatic processing of inspection data

[0438] Uploading data

[0439] The user uploads a CSV file of the test data to the server through the system interface, and the server saves the file in a specified directory.

[0440] Data cleansing and standardization

[0441] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[0442] Automated progress management

[0443] Regular progress capture and recording

[0444] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[0445] Data Visualization and Analysis

[0446] Data Visualization

[0447] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and provided to the user for viewing via their device.

[0448] Prediction of likely test results

[0449] Use of disease prediction models

[0450] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[0451] Specific examples

[0452] Example 1: A concrete example of automated processing of test data

[0453] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[0454] Example 2: Specific example of automated progress management

[0455] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[0456] Example 3: Specific examples of data visualization and analysis

[0457] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[0458] Example 4: Specific example of predicting the likelihood of test results

[0459] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[0460] These features will significantly improve the work efficiency of medical professionals, enabling them to respond quickly and accurately, and are expected to enhance the quality of service provided to patients, leading to improvements in the overall medical process.

[0461] The processing flow will be explained below.

[0462] Automatic processing of inspection data

[0463] Step 1:

[0464] The user uploads the CSV file of the test data to the server through the system interface. The uploaded file is saved in a specified directory on the server.

[0465] Step 2:

[0466] The server reads the uploaded CSV file. Once the reading is complete, it starts cleansing the data, identifying incomplete data (e.g. rows with missing values ​​or outliers) and removing these rows.

[0467] Step 3:

[0468] The server standardizes the cleansed data by calculating the mean and standard deviation for each data column and standardizing each value (subtracting the mean and dividing by the standard deviation).

[0469] Step 4:

[0470] The standardized data is saved as a new CSV file on the storage device. After the clean data has been saved, the save path is recorded in the log.

[0471] Automated progress management

[0472] Step 1:

[0473] The server obtains the current inspection progress status at regular intervals (for example, every 60 seconds), which includes information such as the inspection progress status and the degree of completion of processing.

[0474] Step 2:

[0475] The server writes the obtained progress status to a file in text format. Specifically, it appends information combining the current timestamp and progress status to the progress_status.txt file.

[0476] Data Visualization and Analysis

[0477] Step 1:

[0478] The server loads the previously saved clean data file (e.g., cleaned_exam_data.csv) and prepares the data for visualization.

[0479] Step 2:

[0480] The server visualizes the data, specifically generating time series graphs and adding appropriate labels and titles. A dedicated library is used to generate the graphs.

[0481] Step 3:

[0482] Save the generated graph as an image file, for example, as exam_data_visualization.png, and record the save path in the log.

[0483] Prediction of likely test results

[0484] Step 1:

[0485] The server loads a pre-trained predictive model (e.g., disease_prediction_model.joblib) that is used to predict the likelihood of disease based on laboratory data.

[0486] Step 2:

[0487] The server inputs the clean data into the predictive model and performs disease prediction: it feeds each row of data into the model and gets the predicted disease outcome.

[0488] Step 3:

[0489] The prediction results are appended to the test data. The appended data is saved as a new CSV file (e.g., exam_data_with_predictions.csv), and the save path is recorded in the log.

[0490] Step 4:

[0491] The user views the prediction results through a terminal, and uses the system interface to display the saved data with prediction results and use them as a reference for diagnosis.

[0492] Through these steps, the system efficiently realizes automatic processing of test data, automated progress management, data visualization and analysis, and probable prediction of test results.

[0493] Example 1

[0494] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0495] In the medical field, management and analysis of test data is often done manually, resulting in reduced efficiency and potential for errors. It is also difficult to grasp progress, which can lead to delayed diagnoses. Furthermore, while there is a need for automated visualization of test data and disease prediction, there is a problem that no system exists that comprehensively solves these issues.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0497] In this invention, the server includes: a means for a user to upload test data; a means for saving the test data in a specified directory; a means for cleansing the test data to remove incomplete data; a means for standardizing the cleansed test data; a means for saving the standardized data on a recording medium; a means for acquiring and recording progress status at regular time intervals; a means for displaying the acquired progress status in real time; a means for updating and managing the recorded progress status information; a means for generating a time-series graph based on the cleansed and standardized test data; a means for saving the graph as an image file; a means for predicting a disease name from the test results using a pre-trained prediction model; and a means for adding and recording the prediction results to the test data. This automates the management and analysis of test data, improves work efficiency, and reduces errors. Furthermore, real-time progress monitoring is possible, resulting in faster and more accurate diagnoses.

[0498] "Test data" means electronic records of information containing the results of medical tests.

[0499] The "means for users to upload test data" refers to an interface that allows users to send test data to the system via their terminals.

[0500] The "means for saving test data in a specified directory" is a function for saving test data received by the server in a specific folder.

[0501] The "means for cleansing test data and removing incomplete data" refers to a process for analyzing uploaded test data and removing missing or erroneous data.

[0502] "Means to standardize cleansed laboratory data" refers to the process of converting data to a unified scale to ensure consistency when inputting data into analytical and predictive models.

[0503] The "means for storing standardized data on a recording medium" is a function for storing test data in a digital format after standardization has been completed.

[0504] The "means for acquiring and recording the progress status at regular intervals" is a system function in which the server periodically monitors the progress of the inspection work and records the information.

[0505] The "means for displaying the acquired progress status in real time" is an interface that displays the recorded progress information so that the user can monitor it in real time.

[0506] The "means for updating and managing recorded progress status information" is a system function that appropriately updates progress information and corrects or supplements it as necessary.

[0507] The "means for generating a time series graph based on cleansed and standardized test data" is a function for creating a graph that visually represents data fluctuations based on processed test data.

[0508] The "means for saving the graph as an image file" is a function for saving the generated time series graph in image format.

[0509] "Means for predicting disease names from test results using a pre-trained predictive model" is a function that predicts diseases from new test data using a model trained on past data.

[0510] The "means for adding and recording prediction results to test data" is a function for adding the predicted disease name to existing test data and resaving it.

[0511] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system aims to improve the work efficiency of medical professionals by analyzing and managing test data uploaded by users on a server.

[0512] System Configuration

[0513] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[0514] Hardware and software used

[0515] The server is a computer system equipped with a powerful processor and sufficient memory, running a Linux-based OS and Apache HTTP Server, and uses an SQL-based database (e.g., MySQL) to store data.

[0516] The terminal is a PC or tablet operated by the user, and the system is accessed through a web browser such as Chrome or Firefox. The web interface is developed using HTML, CSS, and JavaScript.

[0517] Explaining program processing in natural language

[0518] 1. User uploads inspection data

[0519] Through the system interface, users upload test data CSV files to the server, which then stores the files in a specified directory.

[0520] 2. The server performs data cleansing and standardization

[0521] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[0522] 3. The server periodically acquires and records the progress

[0523] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[0524] 4. The server visualizes the data

[0525] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and made available to users via their devices.

[0526] 5. The server performs disease prediction

[0527] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[0528] Specific examples

[0529] Example 1: A concrete example of automated processing of test data

[0530] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[0531] Example 2: Specific example of automated progress management

[0532] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[0533] Example 3: A concrete example of data visualization and analysis

[0534] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[0535] Example 4: Specific example of predicting the likelihood of test results

[0536] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[0537] Prompt Sentence Examples

[0538] Upload the following test data for cleansing, standardization, and disease prediction.

[0539] Filename: exam_data.csv

[0540] This prompt causes the system to automatically begin processing the test data and generate results.

[0541] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0542] System program processing flow

[0543] Processing Steps

[0544] Step 1:

[0545] The user uploads the test data.

[0546] The user opens a web browser on their device and accesses the system interface (HTML form). Using the file selection dialog on the interface, they select the CSV file of the examination data (e.g., exam_data.csv) and click the "Upload" button. The server receives this file and saves it in a specified directory (e.g., / data / uploads / ).

[0547] Input: exam_data.csv (file uploaded by user)

[0548] Output: exam_data.csv saved in the specified directory

[0549] Step 2:

[0550] The server performs data cleansing.

[0551] The server reads the CSV files stored in the specified directory, analyzes the data, and removes rows containing incomplete data or missing values. The cleansed data is temporarily stored in temporary variables in memory.

[0552] Input: exam_data.csv in the specified directory

[0553] Output: Cleansed data (in memory) with imperfect data removed

[0554] Step 3:

[0555] The server performs the normalization of the data.

[0556] The server calculates the mean and standard deviation for each numeric item based on the parsed data, then normalizes each data point using these statistics. The normalized data is saved as a new CSV file (e.g., cleaned_exam_data.csv).

[0557] Input: Cleansed data (in memory)

[0558] Output: cleaned_exam_data.csv containing the standardized data

[0559] Step 4:

[0560] The server periodically obtains and records the progress.

[0561] The server checks the inspection progress every 60 seconds and records the progress data in a text file (e.g., progress_status.txt). This information includes the current data processing status and whether or not there were any errors.

[0562] Input: Internal server progress data

[0563] Output: progress_status.txt (latest progress information)

[0564] Step 5:

[0565] The server visualizes the data.

[0566] The server generates a time series graph based on the standardized data. This graph is saved in image format (e.g., PNG format). The graph is generated using the Python matplotlib library.

[0567] Input: cleaned_exam_data.csv (standardized data)

[0568] Output: exam_data_visualization.png (generated graph)

[0569] Step 6:

[0570] The server performs the disease prediction.

[0571] The server loads a pre-trained machine learning model (e.g., Scikit-learn's random forest model). Standardized test data is input into this model to predict the likelihood of disease. The prediction results are appended to the original data and saved as a new CSV file (e.g., exam_data_with_predictions.csv).

[0572] Input: Standardized test data, trained prediction model

[0573] Output: exam_data_with_predictions.csv (data with prediction results)

[0574] Step 7:

[0575] The user views the results.

[0576] Users can access the system interface through their terminal and download or view the prediction result file (e.g., exam_data_with_predictions.csv), allowing them to check the prediction results and use them as a reference for diagnosis.

[0577] Input: exam_data_with_predictions.csv (data with prediction results)

[0578] Output: Prediction result data viewed by the user

[0579] (Application example 1)

[0580] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0581] In modern factories, progress management and anomaly detection in production processes are important issues. However, there are insufficient means to efficiently carry out these management and detection tasks, which requires a lot of time and effort. In addition, it is difficult to quickly notify and respond when an anomaly occurs, which often leads to a decline in productivity and quality. Therefore, there is a need for efficient real-time progress management, anomaly detection, and data visualization.

[0582] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0583] In this invention, the server includes means for converting the inspection data into an electronic format and saving it, means for cleansing the inspection data to remove incomplete data, means for standardizing the cleansed inspection data, means for saving the standardized data on a recording medium, means for acquiring and recording a progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for detecting the possibility of an anomaly occurring using an anomaly detection model, means for notifying a detected anomaly via a smartphone, and means for displaying the data as a graph and saving it as an image file. This enables efficient management of the progress status of the production process, enables rapid response when an anomaly occurs, and is expected to improve productivity and quality.

[0584] "Inspection data" is information that includes measurements and status information related to a factory or manufacturing process.

[0585] "Cleansing" is the process of removing missing or outliers from data and shaping the data.

[0586] "Standardization" is the process of reducing variability in data and converting it to a uniform scale.

[0587] A "recording medium" is a physical or electronic means for storing data.

[0588] "Progress" is information that indicates the progress of work in a factory or manufacturing process.

[0589] "Real-time" means that the latest information is always acquired and displayed in real time.

[0590] "Update management" is the management process for keeping information and data up to date.

[0591] An "anomaly detection model" is a machine learning model for detecting abnormal values ​​in data.

[0592] "Anomaly" refers to a condition or occurrence that is different from the norm, including problems in the manufacturing process.

[0593] A "smartphone" is a mobile device that can connect to the Internet and use multifunctional applications.

[0594] "Graphical display of data" is a method of converting and displaying data in a form that is visually easy to understand.

[0595] An "image file" is a digital file format for storing visual information.

[0596] A "trained predictive model" is a model that has been trained using past data and can predict future states.

[0597] "Progress delay" is a phenomenon that indicates a delay between planned progress and actual progress.

[0598] "Production progress" is information indicating the current progress of work in the manufacturing process.

[0599] The system for realizing this invention is mainly composed of three elements: a server, a terminal, and a user. Each specific processing step will now be described.

[0600] 1. Upload your data

[0601] Users upload data generated by factory robots and manufacturing machines to a server in CSV file format using a smartphone interface.

[0602] 2. Data Cleansing and Standardization

[0603] The server receives the uploaded data, removes incomplete data, reduces the data variability, and converts it to a standard scale.

[0604] 3. Data storage

[0605] The cleansed and standardized data is stored on a storage medium, which is typically a database (e.g., MySQL or PostgreSQL).

[0606] 4. Automated progress management

[0607] The server collects and records the factory progress at regular intervals, and this information is updated in real time and can be viewed by users via a smartphone app.

[0608] 5. Anomaly detection and notification

[0609] The server analyzes the data using a pre-trained anomaly detection model (using TensorFlow or PyTorch) to detect anomalies, and notifies the user of any detected anomalies via their smartphone.

[0610] 6. Data visualization and analysis

[0611] The server generates a time series graph based on the cleansed and standardized data, which is saved as an image file (PNG file) and displayed to the user via a smartphone app.

[0612] 7. Progress forecast using predictive models

[0613] The server uses the trained prediction model to predict possible delays in progress or abnormalities. The prediction results are added to the data and saved again. This allows users to visually check the actual progress and take any necessary measures quickly.

[0614] Specific examples

[0615] For example, if a production line in a factory is behind schedule or if there is a possibility of a machine malfunction, the system will automatically visualize the progress and potential malfunction and warn the user, who can then check the progress in real time via a smartphone app.

[0616] Prompt Sentence Examples

[0617] Examples of prompts for generative AI models include:

[0618] Analyze factory production line data and predict the possibility of anomalies. Visualize progress taking into account production speed and machine status, and generate alerts if delays or anomalies are detected.

[0619] This will enable efficient management of factory production processes and improve productivity and quality.

[0620] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0621] Step 1:

[0622] The user uploads data generated by factory robots and manufacturing machines to the server in CSV file format. Specifically, the user uses a smartphone app to select the desired CSV file from the file selection screen. This input data includes numerical values ​​and status information about the manufacturing process. The server saves the received file in a temporary directory.

[0623] Step 2:

[0624] The server reads the uploaded CSV file and cleanses the data, specifically removing incomplete data and converting it to the required format. The input at this stage is the CSV file and the output is a cleansed DataFrame. The software used is Python and the Pandas library.

[0625] Step 3:

[0626] The server standardizes the cleansed data. Specifically, it calculates the mean and standard deviation of each data and standardizes the data. This standardized data is stored in a database. The input is a cleansed data frame, and the output is a standardized data frame. The software used is Python and the Scikit-learn library.

[0627] Step 4:

[0628] The server obtains the progress status at regular intervals and records that information. Specifically, the server saves the progress data it obtains from each manufacturing machine by appending it to a text file. The input is the progress data, and the output is a text file containing the progress status. This process is performed periodically using scheduling software.

[0629] Step 5:

[0630] The server displays the acquired progress status in real time. Specifically, it updates and displays the progress status information in real time on a web interface or smartphone app. The input is the progress status data, and the output is the latest progress status displayed on the user interface.

[0631] Step 6:

[0632] The server uses an anomaly detection model to detect possible anomalies in the data. Specifically, it inputs the cleansed and standardized data into the AI ​​model and calculates an anomaly score. The input is the standardized data, and the output is the anomaly score. The software used is TensorFlow or PyTorch.

[0633] Step 7:

[0634] The server notifies the user of detected anomalies via their smartphone. Specifically, if the anomaly score exceeds a threshold, it sends a push notification to the smartphone app. The input is the anomaly score, and the output is a notification message to the user.

[0635] Step 8:

[0636] The server generates a time series graph based on the cleansed and standardized data. Specifically, it generates the graph using Matplotlib or Plotly and saves it as an image file. The input is the standardized data, and the output is a time series graph image.

[0637] Step 9:

[0638] The server uses a trained prediction model to predict the possibility of progress delays or abnormalities. Specifically, it inputs standardized data into the prediction model and obtains a prediction result. The input is the standardized data, and the output is the prediction result. The software used is TensorFlow and PyTorch.

[0639] Step 10:

[0640] The server adds the prediction results to the data and saves them back to the database. Specifically, the obtained prediction results are added to the existing data frame and saved as a new one. The input is the prediction results and the existing data frame, and the output is the updated data frame.

[0641] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0642] This invention is a system that combines automatic processing of medical test data, automated progress management, data visualization and analysis, and probable test result prediction in the medical field with an emotion engine that recognizes the user's emotions. This system uses the emotion engine to monitor the user's emotions and provides appropriate interfaces and feedback according to the user's emotional state, thereby improving the work efficiency of medical professionals and the patient experience.

[0643] System Configuration

[0644] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running predictive models, and operating the emotion engine, while the terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[0645] Introducing the Emotion Engine

[0646] User Emotion Recognition

[0647] The server runs an emotion engine based on the user's voice, facial expressions, text input, etc. to recognize the user's emotional state, and provides appropriate feedback according to the user's emotional state.

[0648] Dynamically changing the display of test data based on emotions

[0649] The server dynamically changes how the test data is displayed based on the user's perceived emotions, for example, by displaying more detailed information and explanations if the user is anxious, to provide a sense of security.

[0650] Emotional state monitoring and collaboration

[0651] The emotion engine continuously monitors the user's emotional state, and the results are recorded in relation to progress, allowing changes in the user's mental state to be tracked.

[0652] Specific examples

[0653] Example 1: Emotion Recognition

[0654] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Based on this information, the server displays an interface that matches the user's mood.

[0655] Example 2: Dynamic data display

[0656] When the user tries to check the test results, the server recognizes that the user is anxious using its emotion engine. In addition to the usual display method, the server displays detailed explanations of the results and reference information to alleviate the user's anxiety.

[0657] Example 3: A concrete example of linking and monitoring emotional states

[0658] The server simultaneously runs an emotion engine to monitor the user's emotional state while the user checks the test data and views the progress. The results are recorded along with the progress so that medical professionals can later understand the user's emotional changes.

[0659] This system not only improves the work efficiency of medical professionals, but also provides services that take into consideration the mental health of patients. By combining it with an emotion engine, it becomes possible to provide responses that are optimized for each individual user, resulting in the provision of higher quality medical services.

[0660] The processing flow will be explained below.

[0661] Embodiment of a system combining emotion engines

[0662] Recognizing user emotions with an emotion engine

[0663] Step 1:

[0664] The user logs into the system.

[0665] The server receives the user's login information and performs authentication.

[0666] Step 2:

[0667] When the user initiates an action, the device's camera and microphone become active.

[0668] The device collects the user's facial expressions and voice data in real time.

[0669] Step 3:

[0670] The server sends the collected facial expression and voice data to the emotion engine.

[0671] The emotion engine uses facial expression analysis and voice analysis algorithms to determine the user's emotional state.

[0672] Step 4:

[0673] The emotion engine determines the user's emotion and returns the result to the server.

[0674] The server receives the data and records the user's current emotional state.

[0675] Dynamically changing the display of test data based on emotions

[0676] Step 1:

[0677] The user performs operations to view the inspection data.

[0678] The terminal sends the request to the server.

[0679] Step 2:

[0680] The server ascertains the user's emotional state.

[0681] Refer to the data of the emotional state determined by the emotion engine.

[0682] Step 3:

[0683] The server changes how the test data is displayed based on the user's emotional state.

[0684] For example, if the user is in an anxious state, detailed explanations or reassuring messages are displayed.

[0685] Step 4:

[0686] The display content is transmitted to the terminal based on the changed display method.

[0687] The terminal displays the test data in a user-readable format.

[0688] Emotional state monitoring and collaboration

[0689] Step 1:

[0690] While the user is using the system, the device continuously collects facial expression and voice data.

[0691] The collected data is periodically sent to a server.

[0692] Step 2:

[0693] The server periodically sends data to the emotion engine to update the user's emotional state.

[0694] The updated emotion data is saved to the server in real time.

[0695] Step 3:

[0696] The server integrates the progress status data and the emotion data, and records the changes in the user's emotional state in association with the progress status.

[0697] The recorded data is later used by medical personnel to ascertain the user's emotional state.

[0698] Step 4:

[0699] When the user exits the system, the server records the final emotional state and ends the session.

[0700] The device will disable the camera and microphone and stop transmitting collected data.

[0701] <<Example>>

[0702] Example 1: User Emotion Recognition

[0703] When a user logs in to the system, the device's camera and microphone collect facial expressions and voice in real time. The server sends this data to an emotion engine to recognize when the user is feeling stressed. Based on the results, the server displays an interface using a theme color that has a relaxing effect on the user.

[0704] Example 2: Dynamic data display based on emotions

[0705] When a user attempts to view a particular test result, the server detects through an emotion engine that the user is feeling anxious, and the device displays the test data with detailed explanations and a message saying, "If you have any questions, please consult your doctor."

[0706] Example 3: Monitoring and recording emotional states

[0707] While the user is checking their progress, the server uses an emotion engine to monitor their emotional state and records the results along with their progress. If the user becomes frustrated, the server notifies medical professionals and prompts them to take appropriate action.

[0708] These steps will enable the system to achieve more flexible and efficient medical data management and provision that takes into account users' emotions.

[0709] Example 2

[0710] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0711] The modern medical field requires efficient processing and management of test data. However, existing systems are unable to adequately address the following: test data cleansing, standardization, storage, real-time display and recording of progress, data visualization, and disease prediction using predictive models. They also lack the ability to recognize the user's emotional state and provide appropriate feedback and interfaces accordingly. This makes it difficult to improve the work efficiency of medical professionals and the psychological satisfaction of patients.

[0712] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for analyzing the user's voice, facial expression, and text input to recognize the user's emotional state, means for dynamically changing the display method of the test data based on the recognized emotional state, and means for continuously monitoring the user's emotional state and recording the results in association with the progress status. This enables efficient processing and management of test data and further enables the provision of appropriate feedback and interfaces according to the user's emotional state. This improves the work efficiency of medical professionals and increases the psychological satisfaction of patients.

[0713] "Test data" refers to information including results and measurements obtained by a user at a medical institution or testing facility.

[0714] "Cleansing" is the process of removing incomplete data and errors from test data and organizing it.

[0715] "Standardization" is the process of aligning cleansed data into a consistent format and units.

[0716] A "recording medium" is a digital storage device or service for storing data.

[0717] "Progress" is information that indicates the progress and current status of medical treatments and examinations.

[0718] "Emotional state" refers to the user's psychological state or mood.

[0719] "Voice analysis" is a technology that analyzes voice data and extracts specific patterns and emotions.

[0720] "Facial expression analysis" is a technology that analyzes facial expressions captured by a camera and recognizes emotions.

[0721] "Text analysis" is the process of analyzing textual information entered by a user to understand their emotions and intent.

[0722] "Dynamic change" is a mechanism that changes the display content and behavior of the system in real time according to specific conditions or situations.

[0723] "Monitoring" is the act of continuously observing a user's state and behavior and collecting data.

[0724] "Feedback" is information or a response provided to a user with the purpose of approving or correcting their behavior.

[0725] This system is specialized for efficient processing and management of medical test data. It automates a series of processes, from test data cleansing to standardization, storage, and sentiment analysis, with the aim of improving the work efficiency of medical professionals and the psychological satisfaction of patients.

[0726] System Configuration

[0727] The system consists of three components: a server, a terminal, and a user. The server processes and stores data, runs predictive models, and operates the emotion engine. The terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[0728] Hardware and software used

[0729] Hardware: Use a server equipped with a high-performance processor and a large amount of memory (e.g., Dell PowerEdge, HP ProLiant) as the server. Use a personal computer or tablet equipped with a camera and microphone as the terminal.

[0730] Software: The server uses "EmotionAPI," "database management system (e.g., MySQL, PostgreSQL)," and "web server (e.g., Apache, Nginx)." The terminal uses technologies such as "WebRTC" and "JavaScript framework (e.g., React.js)."

[0731] Data processing and calculation

[0732] 1. Cleansing: The server receives the uploaded test data and filters out missing or inappropriate data.

[0733] 2. Standardization: The cleansed data is converted into a unified format and stored in a database.

[0734] 3. Emotion recognition: The user's voice, facial expressions, and text input sent from the device are analyzed using the "Emotion API" to recognize the user's emotional state.

[0735] 4. Dynamic Change: The server dynamically changes how the test data is displayed based on the perceived emotional state. For example, if the user is anxious, it provides more information and reassuring feedback.

[0736] 5. Monitoring: The server continuously monitors the user's emotional state and records it in relation to their progress.

[0737] Specific examples

[0738] Example 1: Emotion Recognition

[0739] When a user logs in to the system, the device uses a camera and microphone to capture the user's facial expressions and voice. The server analyzes the data using the "Emotion API" to recognize the user's emotional state. If the user is feeling anxious, the server will provide an interface that provides a sense of security.

[0740] Example 2: Dynamic data display

[0741] When a user attempts to check the results of their test data, the device sends a request to the server. The server recognizes that the user is anxious and sends the results to the device with a detailed explanation and reference information. The device then displays the results to the user.

[0742] Example 3: A concrete example of emotional state monitoring

[0743] The server uses an emotion engine to monitor the user's emotional state while they are reviewing the test data and viewing their progress, and records the results in association with their progress so that medical professionals can review them later.

[0744] Prompt Sentence Examples

[0745] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Then, after reviewing the test data, the server displays the results with detailed explanations and provides feedback to the user to give them a sense of security.

[0746] This allows the system to not only efficiently process and manage test data, but also to provide an interface and feedback that takes user emotions into consideration, thereby improving the work efficiency of medical professionals and increasing the psychological satisfaction of patients.

[0747] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0748] Step 1:

[0749] When a user logs into the system, the terminal takes the user's email address and password as input and sends them to the server. The server queries a database to verify the authentication information. If authentication is successful, the server outputs a login success message to the terminal. The terminal receives this and notifies the user that the login was successful.

[0750] Step 2:

[0751] The device uses a camera and microphone to capture the user's facial expressions and voice. This captured data is sent as input to the server. The server analyzes the received data using the Emotion API to recognize the user's emotional state. Based on this result, the server generates appropriate user interface data and outputs it to the device. The device then displays the received interface data.

[0752] Step 3:

[0753] Users upload test data files using a drag-and-drop interface on their devices, which then send the files as input to the server, which cleanses the received test data and removes incomplete data. The cleansed data is then standardized and stored in a database.

[0754] Step 4:

[0755] The user requests confirmation of test data from the device. The device sends this request as input to the server. The server retrieves the relevant test data from the database and again uses the Emotion API to recognize the user's current emotional state. The server receives the test data and emotional state as input and generates detailed feedback. The device displays this feedback.

[0756] Step 5:

[0757] The device continuously captures the user's voice and facial expressions using a camera and microphone. This periodically captured data is sent as input to a server. The server uses the Emotion API to recognize the user's emotional state and record it in a database. The results are associated with progress and stored in the database. Medical professionals can access the database to check changes in the user's emotional state.

[0758] (Application example 2)

[0759] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0760] Conventional food delivery applications provide menu recommendations and feedback without considering the user's emotional state. As a result, it is difficult to provide appropriate support to users who are feeling stressed or anxious, limiting the quality of the user experience. Furthermore, they are unable to provide a dynamic interface based on the user's emotions, and improvements that would lead to increased user satisfaction are needed.

[0761] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for providing appropriate feedback based on the user's emotions using an emotion engine that recognizes the user's emotional state by analyzing the user's facial expressions and voice, and means for analyzing the user's facial expressions and presenting dietary recommendations based on the analysis. This enables feedback and recommendations that take the user's emotional state into consideration, significantly improving the quality of the user experience.

[0762] "Electronic test data format" refers to the technical means for reading test data, converting the data into a digital format, and storing it electronically.

[0763] "Cleansing" is the process of removing incomplete data and noise from inspection data to improve data quality.

[0764] "Standardization" is a technical means of converting cleansed data into a uniform format that makes it easier to compare and analyze.

[0765] "Storage medium" means a physical or electronic device or system for storing standardized data.

[0766] "Status capture" is the process of periodically reviewing and recording data and project progress.

[0767] "Real-time display" refers to a technical means that instantly reflects progress and data and displays them to the user.

[0768] Update management is the process of keeping recorded progress information up to date and managing changes and additions.

[0769] An "emotion engine" is an algorithm or technology that analyzes and recognizes a user's emotional state from their facial expressions and voice.

[0770] "Feedback" is a response that provides appropriate information or recommendations depending on the user's perceived emotional state.

[0771] "Dietary recommendations" are information that suggests meals and menus that are optimal for the user's emotional state at the time, based on the user's facial expression.

[0772] "Data visualization" is a technical method of displaying inspection data in the form of graphs and charts to make it easier to understand.

[0773] "Dynamic display change" is a technical means for changing the way data is displayed in real time according to the user's emotional state.

[0774] "Emotion monitoring" is the process of continuously tracking a user's emotional state and recording its changes.

[0775] A "predictive model" is an algorithm or technique that uses a pre-trained dataset to predict outcomes from new data.

[0776] A "prompt" is an instruction or command to be input into a generative AI model.

[0777] The present invention provides a system for recognizing a user's emotions in food delivery and providing appropriate feedback and recommendations based on the emotions. Specific embodiments of the system are described below.

[0778] System Configuration

[0779] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running the emotion engine, and operating the predictive model. The terminal provides an interface for users to access and operate the system. Users order food and check feedback.

[0780] Introducing the Emotion Engine

[0781] User Emotion Recognition

[0782] The server uses an image processing library (e.g., OpenCV) and an emotion recognition model (e.g., a model trained with Keras) to analyze the user's facial expressions. Images and videos uploaded by the user are taken, and the server processes them to recognize the user's emotional state.

[0783] Emotion-based dietary recommendations

[0784] The server provides appropriate feedback based on the recognized emotion, suggesting the most suitable menu from a predefined list to provide dietary recommendations according to the user's emotional state. For example, if the user is anxious, it will recommend relaxing meals and herbal teas.

[0785] Emotional state monitoring and collaboration

[0786] The server runs an emotion engine in real time to monitor the user's emotional state when confirming an order, records the results sequentially, and analyzes long-term trends to provide strategies to improve user satisfaction.

[0787] Specific examples

[0788] Examples of facial expression analysis and emotion recognition

[0789] When a user accesses the system, they take a picture of their face using the device's camera and upload it to the server. The server processes the image and recognizes the user's emotional state. Specific examples of prompts are as follows:

[0790] Analyze the image file "user_face.jpg" and recognize the user's emotions.

[0791] Provide appropriate feedback based on predicted emotions.

[0792] Dynamic Data Display and Feedback Example

[0793] When the user tries to confirm their order, the server analyzes the user's emotions and dynamically changes the feedback based on the results. For example, if the user is anxious, a recommendation message such as "Try a menu that has a relaxing effect" will be displayed.

[0794] Hardware and software used

[0795] Hardware: The user's device (such as a smartphone or tablet)

[0796] Software: Image processing libraries (e.g., OpenCV), machine learning frameworks (e.g., Keras), frameworks (e.g., Flask) on the server

[0797] In this way, personalized responses that take into account the user's emotional state can be made, significantly improving the user experience.

[0798] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0799] Step 1:

[0800] A user takes a picture of their face using the device camera and uploads it to the system. The input is the user's face image file, and the output is the image data sent to the server.

[0801] Step 2:

[0802] The server analyzes the received facial image data using the OpenCV library. Specifically, it uses a facial recognition algorithm to detect the face and extract the necessary parts. The input is the user's facial image data, and the output is image data of the facial area.

[0803] Step 3:

[0804] The server inputs the facial image data into an emotion recognition model trained with Keras to predict emotions. The input is the facial image data, and the output is the user's emotional state (e.g., "anxiety," "joy," "anger," etc.).

[0805] Step 4:

[0806] The server selects appropriate feedback and dietary recommendations based on the predicted emotional state. The input is the emotional state, and the output is a feedback message or recommended menu. For example, it performs a specific action such as "if the user is anxious, recommend a relaxing herbal tea."

[0807] Step 5:

[0808] The server sends the selected feedback and recommended menu to the terminal. The input is the feedback message and recommended menu, and the output is the information displayed on the terminal.

[0809] Step 6:

[0810] The terminal displays the feedback messages and recommended menus received from the server to the user. The input is the feedback information from the server, and the output is the information visually displayed to the user.

[0811] Step 7:

[0812] The server records the user's order history and feedback, and monitors changes in their emotional state. The input is the user's order and feedback data, and the output is continuous monitoring data. This allows us to analyze long-term user satisfaction and use it to further improve our services.

[0813] In this way, each processing step is performed sequentially, providing real-time feedback and recommendations to improve the user experience.

[0814] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0815] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0816] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0817] [Third embodiment]

[0818] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0819] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0820] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0821] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0822] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0823] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0824] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0825] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0826] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0827] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0828] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0829] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0830] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system analyzes and manages test data uploaded by users through various processes, improving the work efficiency of medical professionals.

[0831] System Configuration

[0832] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[0833] Automatic processing of inspection data

[0834] Uploading data

[0835] The user uploads a CSV file of the test data to the server through the system interface, and the server saves the file in a specified directory.

[0836] Data cleansing and standardization

[0837] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[0838] Automated progress management

[0839] Regular progress capture and recording

[0840] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[0841] Data Visualization and Analysis

[0842] Data Visualization

[0843] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and provided to the user for viewing via their device.

[0844] Prediction of likely test results

[0845] Use of disease prediction models

[0846] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[0847] Specific examples

[0848] Example 1: A concrete example of automated processing of test data

[0849] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[0850] Example 2: Specific example of automated progress management

[0851] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[0852] Example 3: Specific examples of data visualization and analysis

[0853] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[0854] Example 4: Specific example of predicting the likelihood of test results

[0855] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[0856] These features will significantly improve the work efficiency of medical professionals, enabling them to respond quickly and accurately, and are expected to enhance the quality of service provided to patients, leading to improvements in the overall medical process.

[0857] The processing flow will be explained below.

[0858] Automatic processing of inspection data

[0859] Step 1:

[0860] The user uploads the CSV file of the test data to the server through the system interface. The uploaded file is saved in a specified directory on the server.

[0861] Step 2:

[0862] The server reads the uploaded CSV file. Once the reading is complete, it starts cleansing the data, identifying incomplete data (e.g. rows with missing values ​​or outliers) and removing these rows.

[0863] Step 3:

[0864] The server standardizes the cleansed data by calculating the mean and standard deviation for each data column and standardizing each value (subtracting the mean and dividing by the standard deviation).

[0865] Step 4:

[0866] The standardized data is saved as a new CSV file on the storage device. After the clean data has been saved, the save path is recorded in the log.

[0867] Automated progress management

[0868] Step 1:

[0869] The server obtains the current inspection progress status at regular intervals (for example, every 60 seconds), which includes information such as the inspection progress status and the degree of completion of processing.

[0870] Step 2:

[0871] The server writes the obtained progress status to a file in text format. Specifically, it appends information combining the current timestamp and progress status to the progress_status.txt file.

[0872] Data Visualization and Analysis

[0873] Step 1:

[0874] The server loads the previously saved clean data file (e.g., cleaned_exam_data.csv) and prepares the data for visualization.

[0875] Step 2:

[0876] The server visualizes the data, specifically generating time series graphs and adding appropriate labels and titles. A dedicated library is used to generate the graphs.

[0877] Step 3:

[0878] Save the generated graph as an image file, for example, as exam_data_visualization.png, and record the save path in the log.

[0879] Prediction of likely test results

[0880] Step 1:

[0881] The server loads a pre-trained predictive model (e.g., disease_prediction_model.joblib) that is used to predict the likelihood of disease based on laboratory data.

[0882] Step 2:

[0883] The server inputs the clean data into the predictive model and performs disease prediction: it feeds each row of data into the model and gets the predicted disease outcome.

[0884] Step 3:

[0885] The prediction results are appended to the test data. The appended data is saved as a new CSV file (e.g., exam_data_with_predictions.csv), and the save path is recorded in the log.

[0886] Step 4:

[0887] The user views the prediction results through a terminal, and uses the system interface to display the saved data with prediction results and use them as a reference for diagnosis.

[0888] Through these steps, the system efficiently realizes automatic processing of test data, automated progress management, data visualization and analysis, and probable prediction of test results.

[0889] Example 1

[0890] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0891] In the medical field, management and analysis of test data is often done manually, resulting in reduced efficiency and potential for errors. It is also difficult to grasp progress, which can lead to delayed diagnoses. Furthermore, while there is a need for automated visualization of test data and disease prediction, there is a problem that no system exists that comprehensively solves these issues.

[0892] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0893] In this invention, the server includes: a means for a user to upload test data; a means for saving the test data in a specified directory; a means for cleansing the test data to remove incomplete data; a means for standardizing the cleansed test data; a means for saving the standardized data on a recording medium; a means for acquiring and recording progress status at regular time intervals; a means for displaying the acquired progress status in real time; a means for updating and managing the recorded progress status information; a means for generating a time-series graph based on the cleansed and standardized test data; a means for saving the graph as an image file; a means for predicting a disease name from the test results using a pre-trained prediction model; and a means for adding and recording the prediction results to the test data. This automates the management and analysis of test data, improves work efficiency, and reduces errors. Furthermore, real-time progress monitoring is possible, resulting in faster and more accurate diagnoses.

[0894] "Test data" means electronic records of information containing the results of medical tests.

[0895] The "means for users to upload test data" refers to an interface that allows users to send test data to the system via their terminals.

[0896] The "means for saving test data in a specified directory" is a function for saving test data received by the server in a specific folder.

[0897] The "means for cleansing test data and removing incomplete data" refers to a process for analyzing uploaded test data and removing missing or erroneous data.

[0898] "Means to standardize cleansed laboratory data" refers to the process of converting data to a unified scale to ensure consistency when inputting data into analytical and predictive models.

[0899] The "means for storing standardized data on a recording medium" is a function for storing test data in a digital format after standardization has been completed.

[0900] The "means for acquiring and recording the progress status at regular intervals" is a system function in which the server periodically monitors the progress of the inspection work and records the information.

[0901] The "means for displaying the acquired progress status in real time" is an interface that displays the recorded progress information so that the user can monitor it in real time.

[0902] The "means for updating and managing recorded progress status information" is a system function that appropriately updates progress information and corrects or supplements it as necessary.

[0903] The "means for generating a time series graph based on cleansed and standardized test data" is a function for creating a graph that visually represents data fluctuations based on processed test data.

[0904] The "means for saving the graph as an image file" is a function for saving the generated time series graph in image format.

[0905] "Means for predicting disease names from test results using a pre-trained predictive model" is a function that predicts diseases from new test data using a model trained on past data.

[0906] The "means for adding and recording prediction results to test data" is a function for adding the predicted disease name to existing test data and resaving it.

[0907] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system aims to improve the work efficiency of medical professionals by analyzing and managing test data uploaded by users on a server.

[0908] System Configuration

[0909] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[0910] Hardware and software used

[0911] The server is a computer system equipped with a powerful processor and sufficient memory, running a Linux-based OS and Apache HTTP Server, and uses an SQL-based database (e.g., MySQL) to store data.

[0912] The terminal is a PC or tablet operated by the user, and the system is accessed through a web browser such as Chrome or Firefox. The web interface is developed using HTML, CSS, and JavaScript.

[0913] Explaining program processing in natural language

[0914] 1. User uploads inspection data

[0915] Through the system interface, users upload test data CSV files to the server, which then stores the files in a specified directory.

[0916] 2. The server performs data cleansing and standardization

[0917] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[0918] 3. The server periodically acquires and records the progress

[0919] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[0920] 4. The server visualizes the data

[0921] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and made available to users via their devices.

[0922] 5. The server performs disease prediction

[0923] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[0924] Specific examples

[0925] Example 1: A concrete example of automated processing of test data

[0926] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[0927] Example 2: Specific example of automated progress management

[0928] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[0929] Example 3: A concrete example of data visualization and analysis

[0930] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[0931] Example 4: Specific example of predicting the likelihood of test results

[0932] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[0933] Prompt Sentence Examples

[0934] Upload the following test data for cleansing, standardization, and disease prediction.

[0935] Filename: exam_data.csv

[0936] This prompt causes the system to automatically begin processing the test data and generate results.

[0937] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0938] System program processing flow

[0939] Processing Steps

[0940] Step 1:

[0941] The user uploads the test data.

[0942] The user opens a web browser on their device and accesses the system interface (HTML form). Using the file selection dialog on the interface, they select the CSV file of the examination data (e.g., exam_data.csv) and click the "Upload" button. The server receives this file and saves it in a specified directory (e.g., / data / uploads / ).

[0943] Input: exam_data.csv (file uploaded by user)

[0944] Output: exam_data.csv saved in the specified directory

[0945] Step 2:

[0946] The server performs data cleansing.

[0947] The server reads the CSV files stored in the specified directory, analyzes the data, and removes rows containing incomplete data or missing values. The cleansed data is temporarily stored in temporary variables in memory.

[0948] Input: exam_data.csv in the specified directory

[0949] Output: Cleansed data (in memory) with imperfect data removed

[0950] Step 3:

[0951] The server performs the normalization of the data.

[0952] The server calculates the mean and standard deviation for each numeric item based on the parsed data, then normalizes each data point using these statistics. The normalized data is saved as a new CSV file (e.g., cleaned_exam_data.csv).

[0953] Input: Cleansed data (in memory)

[0954] Output: cleaned_exam_data.csv containing the standardized data

[0955] Step 4:

[0956] The server periodically obtains and records the progress.

[0957] The server checks the inspection progress every 60 seconds and records the progress data in a text file (e.g., progress_status.txt). This information includes the current data processing status and whether or not there were any errors.

[0958] Input: Internal server progress data

[0959] Output: progress_status.txt (latest progress information)

[0960] Step 5:

[0961] The server visualizes the data.

[0962] The server generates a time series graph based on the standardized data. This graph is saved in image format (e.g., PNG format). The graph is generated using the Python matplotlib library.

[0963] Input: cleaned_exam_data.csv (standardized data)

[0964] Output: exam_data_visualization.png (generated graph)

[0965] Step 6:

[0966] The server performs the disease prediction.

[0967] The server loads a pre-trained machine learning model (e.g., Scikit-learn's random forest model). Standardized test data is input into this model to predict the likelihood of disease. The prediction results are appended to the original data and saved as a new CSV file (e.g., exam_data_with_predictions.csv).

[0968] Input: Standardized test data, trained prediction model

[0969] Output: exam_data_with_predictions.csv (data with prediction results)

[0970] Step 7:

[0971] The user views the results.

[0972] Users can access the system interface through their terminal and download or view the prediction result file (e.g., exam_data_with_predictions.csv), allowing them to check the prediction results and use them as a reference for diagnosis.

[0973] Input: exam_data_with_predictions.csv (data with prediction results)

[0974] Output: Prediction result data viewed by the user

[0975] (Application example 1)

[0976] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0977] In modern factories, progress management and anomaly detection in production processes are important issues. However, there are insufficient means to efficiently carry out these management and detection tasks, which requires a lot of time and effort. In addition, it is difficult to quickly notify and respond when an anomaly occurs, which often leads to a decline in productivity and quality. Therefore, there is a need for efficient real-time progress management, anomaly detection, and data visualization.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0979] In this invention, the server includes means for converting the inspection data into an electronic format and saving it, means for cleansing the inspection data to remove incomplete data, means for standardizing the cleansed inspection data, means for saving the standardized data on a recording medium, means for acquiring and recording a progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for detecting the possibility of an anomaly occurring using an anomaly detection model, means for notifying a detected anomaly via a smartphone, and means for displaying the data as a graph and saving it as an image file. This enables efficient management of the progress status of the production process, enables rapid response when an anomaly occurs, and is expected to improve productivity and quality.

[0980] "Inspection data" is information that includes measurements and status information related to a factory or manufacturing process.

[0981] "Cleansing" is the process of removing missing or outliers from data and shaping the data.

[0982] "Standardization" is the process of reducing variability in data and converting it to a uniform scale.

[0983] A "recording medium" is a physical or electronic means for storing data.

[0984] "Progress" is information that indicates the progress of work in a factory or manufacturing process.

[0985] "Real-time" means that the latest information is always acquired and displayed in real time.

[0986] "Update management" is the management process for keeping information and data up to date.

[0987] An "anomaly detection model" is a machine learning model for detecting abnormal values ​​in data.

[0988] "Anomaly" refers to a condition or occurrence that is different from the norm, including problems in the manufacturing process.

[0989] A "smartphone" is a mobile device that can connect to the Internet and use multifunctional applications.

[0990] "Graphical display of data" is a method of converting and displaying data in a form that is visually easy to understand.

[0991] An "image file" is a digital file format for storing visual information.

[0992] A "trained predictive model" is a model that has been trained using past data and can predict future states.

[0993] "Progress delay" is a phenomenon that indicates a delay between planned progress and actual progress.

[0994] "Production progress" is information indicating the current progress of work in the manufacturing process.

[0995] The system for realizing this invention is mainly composed of three elements: a server, a terminal, and a user. Each specific processing step will now be described.

[0996] 1. Upload your data

[0997] Users upload data generated by factory robots and manufacturing machines to a server in CSV file format using a smartphone interface.

[0998] 2. Data Cleansing and Standardization

[0999] The server receives the uploaded data, removes incomplete data, reduces the data variability, and converts it to a standard scale.

[1000] 3. Data storage

[1001] The cleansed and standardized data is stored on a storage medium, which is typically a database (e.g., MySQL or PostgreSQL).

[1002] 4. Automated progress management

[1003] The server collects and records the factory progress at regular intervals, and this information is updated in real time and can be viewed by users via a smartphone app.

[1004] 5. Anomaly detection and notification

[1005] The server analyzes the data using a pre-trained anomaly detection model (using TensorFlow or PyTorch) to detect anomalies, and notifies the user of any detected anomalies via their smartphone.

[1006] 6. Data visualization and analysis

[1007] The server generates a time series graph based on the cleansed and standardized data, which is saved as an image file (PNG file) and displayed to the user via a smartphone app.

[1008] 7. Progress forecast using predictive models

[1009] The server uses the trained prediction model to predict possible delays in progress or abnormalities. The prediction results are added to the data and saved again. This allows users to visually check the actual progress and take any necessary measures quickly.

[1010] Specific examples

[1011] For example, if a production line in a factory is behind schedule or if there is a possibility of a machine malfunction, the system will automatically visualize the progress and potential malfunction and warn the user, who can then check the progress in real time via a smartphone app.

[1012] Prompt Sentence Examples

[1013] Examples of prompts for generative AI models include:

[1014] Analyze factory production line data and predict the possibility of anomalies. Visualize progress taking into account production speed and machine status, and generate alerts if delays or anomalies are detected.

[1015] This will enable efficient management of factory production processes and improve productivity and quality.

[1016] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1017] Step 1:

[1018] The user uploads data generated by factory robots and manufacturing machines to the server in CSV file format. Specifically, the user uses a smartphone app to select the desired CSV file from the file selection screen. This input data includes numerical values ​​and status information about the manufacturing process. The server saves the received file in a temporary directory.

[1019] Step 2:

[1020] The server reads the uploaded CSV file and cleanses the data, specifically removing incomplete data and converting it to the required format. The input at this stage is the CSV file and the output is a cleansed DataFrame. The software used is Python and the Pandas library.

[1021] Step 3:

[1022] The server standardizes the cleansed data. Specifically, it calculates the mean and standard deviation of each data and standardizes the data. This standardized data is stored in a database. The input is a cleansed data frame, and the output is a standardized data frame. The software used is Python and the Scikit-learn library.

[1023] Step 4:

[1024] The server obtains the progress status at regular intervals and records that information. Specifically, the server saves the progress data it obtains from each manufacturing machine by appending it to a text file. The input is the progress data, and the output is a text file containing the progress status. This process is performed periodically using scheduling software.

[1025] Step 5:

[1026] The server displays the acquired progress status in real time. Specifically, it updates and displays the progress status information in real time on a web interface or smartphone app. The input is the progress status data, and the output is the latest progress status displayed on the user interface.

[1027] Step 6:

[1028] The server uses an anomaly detection model to detect possible anomalies in the data. Specifically, it inputs the cleansed and standardized data into the AI ​​model and calculates an anomaly score. The input is the standardized data, and the output is the anomaly score. The software used is TensorFlow or PyTorch.

[1029] Step 7:

[1030] The server notifies the user of detected anomalies via their smartphone. Specifically, if the anomaly score exceeds a threshold, it sends a push notification to the smartphone app. The input is the anomaly score, and the output is a notification message to the user.

[1031] Step 8:

[1032] The server generates a time series graph based on the cleansed and standardized data. Specifically, it generates the graph using Matplotlib or Plotly and saves it as an image file. The input is the standardized data, and the output is a time series graph image.

[1033] Step 9:

[1034] The server uses a trained prediction model to predict the possibility of progress delays or abnormalities. Specifically, it inputs standardized data into the prediction model and obtains a prediction result. The input is the standardized data, and the output is the prediction result. The software used is TensorFlow and PyTorch.

[1035] Step 10:

[1036] The server adds the prediction results to the data and saves them back to the database. Specifically, the obtained prediction results are added to the existing data frame and saved as a new one. The input is the prediction results and the existing data frame, and the output is the updated data frame.

[1037] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1038] This invention is a system that combines automatic processing of medical test data, automated progress management, data visualization and analysis, and probable test result prediction in the medical field with an emotion engine that recognizes the user's emotions. This system uses the emotion engine to monitor the user's emotions and provides appropriate interfaces and feedback according to the user's emotional state, thereby improving the work efficiency of medical professionals and the patient experience.

[1039] System Configuration

[1040] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running predictive models, and operating the emotion engine, while the terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[1041] Introducing the Emotion Engine

[1042] User Emotion Recognition

[1043] The server runs an emotion engine based on the user's voice, facial expressions, text input, etc. to recognize the user's emotional state, and provides appropriate feedback according to the user's emotional state.

[1044] Dynamically changing the display of test data based on emotions

[1045] The server dynamically changes how the test data is displayed based on the user's perceived emotions, for example, by displaying more detailed information and explanations if the user is anxious, to provide a sense of security.

[1046] Emotional state monitoring and collaboration

[1047] The emotion engine continuously monitors the user's emotional state, and the results are recorded in relation to progress, allowing changes in the user's mental state to be tracked.

[1048] Specific examples

[1049] Example 1: Emotion Recognition

[1050] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Based on this information, the server displays an interface that matches the user's mood.

[1051] Example 2: Dynamic data display

[1052] When the user tries to check the test results, the server recognizes that the user is anxious using its emotion engine. In addition to the usual display method, the server displays detailed explanations of the results and reference information to alleviate the user's anxiety.

[1053] Example 3: A concrete example of linking and monitoring emotional states

[1054] The server simultaneously runs an emotion engine to monitor the user's emotional state while the user checks the test data and views the progress. The results are recorded along with the progress so that medical professionals can later understand the user's emotional changes.

[1055] This system not only improves the work efficiency of medical professionals, but also provides services that take into consideration the mental health of patients. By combining it with an emotion engine, it becomes possible to provide responses that are optimized for each individual user, resulting in the provision of higher quality medical services.

[1056] The processing flow will be explained below.

[1057] Embodiment of a system combining emotion engines

[1058] Recognizing user emotions with an emotion engine

[1059] Step 1:

[1060] The user logs into the system.

[1061] The server receives the user's login information and performs authentication.

[1062] Step 2:

[1063] When the user initiates an action, the device's camera and microphone become active.

[1064] The device collects the user's facial expressions and voice data in real time.

[1065] Step 3:

[1066] The server sends the collected facial expression and voice data to the emotion engine.

[1067] The emotion engine uses facial expression analysis and voice analysis algorithms to determine the user's emotional state.

[1068] Step 4:

[1069] The emotion engine determines the user's emotion and returns the result to the server.

[1070] The server receives the data and records the user's current emotional state.

[1071] Dynamically changing the display of test data based on emotions

[1072] Step 1:

[1073] The user performs operations to view the inspection data.

[1074] The terminal sends the request to the server.

[1075] Step 2:

[1076] The server ascertains the user's emotional state.

[1077] Refer to the data of the emotional state determined by the emotion engine.

[1078] Step 3:

[1079] The server changes how the test data is displayed based on the user's emotional state.

[1080] For example, if the user is in an anxious state, detailed explanations or reassuring messages are displayed.

[1081] Step 4:

[1082] The display content is transmitted to the terminal based on the changed display method.

[1083] The terminal displays the test data in a user-readable format.

[1084] Emotional state monitoring and collaboration

[1085] Step 1:

[1086] While the user is using the system, the device continuously collects facial expression and voice data.

[1087] The collected data is periodically sent to a server.

[1088] Step 2:

[1089] The server periodically sends data to the emotion engine to update the user's emotional state.

[1090] The updated emotion data is saved to the server in real time.

[1091] Step 3:

[1092] The server integrates the progress status data and the emotion data, and records the changes in the user's emotional state in association with the progress status.

[1093] The recorded data is later used by medical personnel to ascertain the user's emotional state.

[1094] Step 4:

[1095] When the user exits the system, the server records the final emotional state and ends the session.

[1096] The device will disable the camera and microphone and stop transmitting collected data.

[1097] <<Example>>

[1098] Example 1: User Emotion Recognition

[1099] When a user logs in to the system, the device's camera and microphone collect facial expressions and voice in real time. The server sends this data to an emotion engine to recognize when the user is feeling stressed. Based on the results, the server displays an interface using a theme color that has a relaxing effect on the user.

[1100] Example 2: Dynamic data display based on emotions

[1101] When a user attempts to view a particular test result, the server detects through an emotion engine that the user is feeling anxious, and the device displays the test data with detailed explanations and a message saying, "If you have any questions, please consult your doctor."

[1102] Example 3: Monitoring and recording emotional states

[1103] While the user is checking their progress, the server uses an emotion engine to monitor their emotional state and records the results along with their progress. If the user becomes frustrated, the server notifies medical professionals and prompts them to take appropriate action.

[1104] These steps will enable the system to achieve more flexible and efficient medical data management and provision that takes into account users' emotions.

[1105] Example 2

[1106] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1107] The modern medical field requires efficient processing and management of test data. However, existing systems are unable to adequately address the following: test data cleansing, standardization, storage, real-time display and recording of progress, data visualization, and disease prediction using predictive models. They also lack the ability to recognize the user's emotional state and provide appropriate feedback and interfaces accordingly. This makes it difficult to improve the work efficiency of medical professionals and the psychological satisfaction of patients.

[1108] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for analyzing the user's voice, facial expression, and text input to recognize the user's emotional state, means for dynamically changing the display method of the test data based on the recognized emotional state, and means for continuously monitoring the user's emotional state and recording the results in association with the progress status. This enables efficient processing and management of test data and further enables the provision of appropriate feedback and interfaces according to the user's emotional state. This improves the work efficiency of medical professionals and increases the psychological satisfaction of patients.

[1109] "Test data" refers to information including results and measurements obtained by a user at a medical institution or testing facility.

[1110] "Cleansing" is the process of removing incomplete data and errors from test data and organizing it.

[1111] "Standardization" is the process of aligning cleansed data into a consistent format and units.

[1112] A "recording medium" is a digital storage device or service for storing data.

[1113] "Progress" is information that indicates the progress and current status of medical treatments and examinations.

[1114] "Emotional state" refers to the user's psychological state or mood.

[1115] "Voice analysis" is a technology that analyzes voice data and extracts specific patterns and emotions.

[1116] "Facial expression analysis" is a technology that analyzes facial expressions captured by a camera and recognizes emotions.

[1117] "Text analysis" is the process of analyzing textual information entered by a user to understand their emotions and intent.

[1118] "Dynamic change" is a mechanism that changes the display content and behavior of the system in real time according to specific conditions or situations.

[1119] "Monitoring" is the act of continuously observing a user's state and behavior and collecting data.

[1120] "Feedback" is information or a response provided to a user with the purpose of approving or correcting their behavior.

[1121] This system is specialized for efficient processing and management of medical test data. It automates a series of processes, from test data cleansing to standardization, storage, and sentiment analysis, with the aim of improving the work efficiency of medical professionals and the psychological satisfaction of patients.

[1122] System Configuration

[1123] The system consists of three components: a server, a terminal, and a user. The server processes and stores data, runs predictive models, and operates the emotion engine. The terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[1124] Hardware and software used

[1125] Hardware: Use a server equipped with a high-performance processor and a large amount of memory (e.g., Dell PowerEdge, HP ProLiant) as the server. Use a personal computer or tablet equipped with a camera and microphone as the terminal.

[1126] Software: The server uses "EmotionAPI," "database management system (e.g., MySQL, PostgreSQL)," and "web server (e.g., Apache, Nginx)." The terminal uses technologies such as "WebRTC" and "JavaScript framework (e.g., React.js)."

[1127] Data processing and calculation

[1128] 1. Cleansing: The server receives the uploaded test data and filters out missing or inappropriate data.

[1129] 2. Standardization: The cleansed data is converted into a unified format and stored in a database.

[1130] 3. Emotion recognition: The user's voice, facial expressions, and text input sent from the device are analyzed using the "Emotion API" to recognize the user's emotional state.

[1131] 4. Dynamic Change: The server dynamically changes how the test data is displayed based on the perceived emotional state. For example, if the user is anxious, it provides more information and reassuring feedback.

[1132] 5. Monitoring: The server continuously monitors the user's emotional state and records it in relation to their progress.

[1133] Specific examples

[1134] Example 1: Emotion Recognition

[1135] When a user logs in to the system, the device uses a camera and microphone to capture the user's facial expressions and voice. The server analyzes the data using the "Emotion API" to recognize the user's emotional state. If the user is feeling anxious, the server will provide an interface that provides a sense of security.

[1136] Example 2: Dynamic data display

[1137] When a user attempts to check the results of their test data, the device sends a request to the server. The server recognizes that the user is anxious and sends the results to the device with a detailed explanation and reference information. The device then displays the results to the user.

[1138] Example 3: A concrete example of emotional state monitoring

[1139] The server uses an emotion engine to monitor the user's emotional state while they are reviewing the test data and viewing their progress, and records the results in association with their progress so that medical professionals can review them later.

[1140] Prompt Sentence Examples

[1141] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Then, after reviewing the test data, the server displays the results with detailed explanations and provides feedback to the user to give them a sense of security.

[1142] This allows the system to not only efficiently process and manage test data, but also to provide an interface and feedback that takes user emotions into consideration, thereby improving the work efficiency of medical professionals and increasing the psychological satisfaction of patients.

[1143] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1144] Step 1:

[1145] When a user logs into the system, the terminal takes the user's email address and password as input and sends them to the server. The server queries a database to verify the authentication information. If authentication is successful, the server outputs a login success message to the terminal. The terminal receives this and notifies the user that the login was successful.

[1146] Step 2:

[1147] The device uses a camera and microphone to capture the user's facial expressions and voice. This captured data is sent as input to the server. The server analyzes the received data using the Emotion API to recognize the user's emotional state. Based on this result, the server generates appropriate user interface data and outputs it to the device. The device then displays the received interface data.

[1148] Step 3:

[1149] Users upload test data files using a drag-and-drop interface on their devices, which then send the files as input to the server, which cleanses the received test data and removes incomplete data. The cleansed data is then standardized and stored in a database.

[1150] Step 4:

[1151] The user requests confirmation of test data from the device. The device sends this request as input to the server. The server retrieves the relevant test data from the database and again uses the Emotion API to recognize the user's current emotional state. The server receives the test data and emotional state as input and generates detailed feedback. The device displays this feedback.

[1152] Step 5:

[1153] The device continuously captures the user's voice and facial expressions using a camera and microphone. This periodically captured data is sent as input to a server. The server uses the Emotion API to recognize the user's emotional state and record it in a database. The results are associated with progress and stored in the database. Medical professionals can access the database to check changes in the user's emotional state.

[1154] (Application example 2)

[1155] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1156] Conventional food delivery applications provide menu recommendations and feedback without considering the user's emotional state. As a result, it is difficult to provide appropriate support to users who are feeling stressed or anxious, limiting the quality of the user experience. Furthermore, they are unable to provide a dynamic interface based on the user's emotions, and improvements that would lead to increased user satisfaction are needed.

[1157] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for providing appropriate feedback based on the user's emotions using an emotion engine that recognizes the user's emotional state by analyzing the user's facial expressions and voice, and means for analyzing the user's facial expressions and presenting dietary recommendations based on the analysis. This enables feedback and recommendations that take the user's emotional state into consideration, significantly improving the quality of the user experience.

[1158] "Electronic test data format" refers to the technical means for reading test data, converting the data into a digital format, and storing it electronically.

[1159] "Cleansing" is the process of removing incomplete data and noise from inspection data to improve data quality.

[1160] "Standardization" is a technical means of converting cleansed data into a uniform format that makes it easier to compare and analyze.

[1161] "Storage medium" means a physical or electronic device or system for storing standardized data.

[1162] "Status capture" is the process of periodically reviewing and recording data and project progress.

[1163] "Real-time display" refers to a technical means that instantly reflects progress and data and displays them to the user.

[1164] Update management is the process of keeping recorded progress information up to date and managing changes and additions.

[1165] An "emotion engine" is an algorithm or technology that analyzes and recognizes a user's emotional state from their facial expressions and voice.

[1166] "Feedback" is a response that provides appropriate information or recommendations depending on the user's perceived emotional state.

[1167] "Dietary recommendations" are information that suggests meals and menus that are optimal for the user's emotional state at the time, based on the user's facial expression.

[1168] "Data visualization" is a technical method of displaying inspection data in the form of graphs and charts to make it easier to understand.

[1169] "Dynamic display change" is a technical means for changing the way data is displayed in real time according to the user's emotional state.

[1170] "Emotion monitoring" is the process of continuously tracking a user's emotional state and recording its changes.

[1171] A "predictive model" is an algorithm or technique that uses a pre-trained dataset to predict outcomes from new data.

[1172] A "prompt" is an instruction or command to be input into a generative AI model.

[1173] The present invention provides a system for recognizing a user's emotions in food delivery and providing appropriate feedback and recommendations based on the emotions. Specific embodiments of the system are described below.

[1174] System Configuration

[1175] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running the emotion engine, and operating the predictive model. The terminal provides an interface for users to access and operate the system. Users order food and check feedback.

[1176] Introducing the Emotion Engine

[1177] User Emotion Recognition

[1178] The server uses an image processing library (e.g., OpenCV) and an emotion recognition model (e.g., a model trained with Keras) to analyze the user's facial expressions. Images and videos uploaded by the user are taken, and the server processes them to recognize the user's emotional state.

[1179] Emotion-based dietary recommendations

[1180] The server provides appropriate feedback based on the recognized emotion, suggesting the most suitable menu from a predefined list to provide dietary recommendations according to the user's emotional state. For example, if the user is anxious, it will recommend relaxing meals and herbal teas.

[1181] Emotional state monitoring and collaboration

[1182] The server runs an emotion engine in real time to monitor the user's emotional state when confirming an order, records the results sequentially, and analyzes long-term trends to provide strategies to improve user satisfaction.

[1183] Specific examples

[1184] Examples of facial expression analysis and emotion recognition

[1185] When a user accesses the system, they take a picture of their face using the device's camera and upload it to the server. The server processes the image and recognizes the user's emotional state. Specific examples of prompts are as follows:

[1186] Analyze the image file "user_face.jpg" and recognize the user's emotions.

[1187] Provide appropriate feedback based on predicted emotions.

[1188] Dynamic Data Display and Feedback Example

[1189] When the user tries to confirm their order, the server analyzes the user's emotions and dynamically changes the feedback based on the results. For example, if the user is anxious, a recommendation message such as "Try a menu that has a relaxing effect" will be displayed.

[1190] Hardware and software used

[1191] Hardware: The user's device (such as a smartphone or tablet)

[1192] Software: Image processing libraries (e.g., OpenCV), machine learning frameworks (e.g., Keras), frameworks (e.g., Flask) on the server

[1193] In this way, personalized responses that take into account the user's emotional state can be made, significantly improving the user experience.

[1194] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1195] Step 1:

[1196] A user takes a picture of their face using the device camera and uploads it to the system. The input is the user's face image file, and the output is the image data sent to the server.

[1197] Step 2:

[1198] The server analyzes the received facial image data using the OpenCV library. Specifically, it uses a facial recognition algorithm to detect the face and extract the necessary parts. The input is the user's facial image data, and the output is image data of the facial area.

[1199] Step 3:

[1200] The server inputs the facial image data into an emotion recognition model trained with Keras to predict emotions. The input is the facial image data, and the output is the user's emotional state (e.g., "anxiety," "joy," "anger," etc.).

[1201] Step 4:

[1202] The server selects appropriate feedback and dietary recommendations based on the predicted emotional state. The input is the emotional state, and the output is a feedback message or recommended menu. For example, it performs a specific action such as "if the user is anxious, recommend a relaxing herbal tea."

[1203] Step 5:

[1204] The server sends the selected feedback and recommended menu to the terminal. The input is the feedback message and recommended menu, and the output is the information displayed on the terminal.

[1205] Step 6:

[1206] The terminal displays the feedback messages and recommended menus received from the server to the user. The input is the feedback information from the server, and the output is the information visually displayed to the user.

[1207] Step 7:

[1208] The server records the user's order history and feedback, and monitors changes in their emotional state. The input is the user's order and feedback data, and the output is continuous monitoring data. This allows us to analyze long-term user satisfaction and use it to further improve our services.

[1209] In this way, each processing step is performed sequentially, providing real-time feedback and recommendations to improve the user experience.

[1210] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1211] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1212] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1213] [Fourth embodiment]

[1214] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1215] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1216] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1217] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1218] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1220] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1221] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1222] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1223] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1224] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1225] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1226] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1227] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system analyzes and manages test data uploaded by users through various processes, improving the work efficiency of medical professionals.

[1228] System Configuration

[1229] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[1230] Automatic processing of inspection data

[1231] Uploading data

[1232] The user uploads a CSV file of the test data to the server through the system interface, and the server saves the file in a specified directory.

[1233] Data cleansing and standardization

[1234] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[1235] Automated progress management

[1236] Regular progress capture and recording

[1237] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[1238] Data Visualization and Analysis

[1239] Data Visualization

[1240] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and provided to the user for viewing via their device.

[1241] Prediction of likely test results

[1242] Use of disease prediction models

[1243] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[1244] Specific examples

[1245] Example 1: A concrete example of automated processing of test data

[1246] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[1247] Example 2: Specific example of automated progress management

[1248] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[1249] Example 3: Specific examples of data visualization and analysis

[1250] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[1251] Example 4: Specific example of predicting the likelihood of test results

[1252] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[1253] These features will significantly improve the work efficiency of medical professionals, enabling them to respond quickly and accurately, and are expected to enhance the quality of service provided to patients, leading to improvements in the overall medical process.

[1254] The processing flow will be explained below.

[1255] Automatic processing of inspection data

[1256] Step 1:

[1257] The user uploads the CSV file of the test data to the server through the system interface. The uploaded file is saved in a specified directory on the server.

[1258] Step 2:

[1259] The server reads the uploaded CSV file. Once the reading is complete, it starts cleansing the data, identifying incomplete data (e.g. rows with missing values ​​or outliers) and removing these rows.

[1260] Step 3:

[1261] The server standardizes the cleansed data by calculating the mean and standard deviation for each data column and standardizing each value (subtracting the mean and dividing by the standard deviation).

[1262] Step 4:

[1263] The standardized data is saved as a new CSV file on the storage device. After the clean data has been saved, the save path is recorded in the log.

[1264] Automated progress management

[1265] Step 1:

[1266] The server obtains the current inspection progress status at regular intervals (for example, every 60 seconds), which includes information such as the inspection progress status and the degree of completion of processing.

[1267] Step 2:

[1268] The server writes the obtained progress status to a file in text format. Specifically, it appends information combining the current timestamp and progress status to the progress_status.txt file.

[1269] Data Visualization and Analysis

[1270] Step 1:

[1271] The server loads the previously saved clean data file (e.g., cleaned_exam_data.csv) and prepares the data for visualization.

[1272] Step 2:

[1273] The server visualizes the data, specifically generating time series graphs and adding appropriate labels and titles. A dedicated library is used to generate the graphs.

[1274] Step 3:

[1275] Save the generated graph as an image file, for example, as exam_data_visualization.png, and record the save path in the log.

[1276] Prediction of likely test results

[1277] Step 1:

[1278] The server loads a pre-trained predictive model (e.g., disease_prediction_model.joblib) that is used to predict the likelihood of disease based on laboratory data.

[1279] Step 2:

[1280] The server inputs the clean data into the predictive model and performs disease prediction: it feeds each row of data into the model and gets the predicted disease outcome.

[1281] Step 3:

[1282] The prediction results are appended to the test data. The appended data is saved as a new CSV file (e.g., exam_data_with_predictions.csv), and the save path is recorded in the log.

[1283] Step 4:

[1284] The user views the prediction results through a terminal, and uses the system interface to display the saved data with prediction results and use them as a reference for diagnosis.

[1285] Through these steps, the system efficiently realizes automatic processing of test data, automated progress management, data visualization and analysis, and probable prediction of test results.

[1286] Example 1

[1287] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1288] In the medical field, management and analysis of test data is often done manually, resulting in reduced efficiency and potential for errors. It is also difficult to grasp progress, which can lead to delayed diagnoses. Furthermore, while there is a need for automated visualization of test data and disease prediction, there is a problem that no system exists that comprehensively solves these issues.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1290] In this invention, the server includes: a means for a user to upload test data; a means for saving the test data in a specified directory; a means for cleansing the test data to remove incomplete data; a means for standardizing the cleansed test data; a means for saving the standardized data on a recording medium; a means for acquiring and recording progress status at regular time intervals; a means for displaying the acquired progress status in real time; a means for updating and managing the recorded progress status information; a means for generating a time-series graph based on the cleansed and standardized test data; a means for saving the graph as an image file; a means for predicting a disease name from the test results using a pre-trained prediction model; and a means for adding and recording the prediction results to the test data. This automates the management and analysis of test data, improves work efficiency, and reduces errors. Furthermore, real-time progress monitoring is possible, resulting in faster and more accurate diagnoses.

[1291] "Test data" means electronic records of information containing the results of medical tests.

[1292] The "means for users to upload test data" refers to an interface that allows users to send test data to the system via their terminals.

[1293] The "means for saving test data in a specified directory" is a function for saving test data received by the server in a specific folder.

[1294] The "means for cleansing test data and removing incomplete data" refers to a process for analyzing uploaded test data and removing missing or erroneous data.

[1295] "Means to standardize cleansed laboratory data" refers to the process of converting data to a unified scale to ensure consistency when inputting data into analytical and predictive models.

[1296] The "means for storing standardized data on a recording medium" is a function for storing test data in a digital format after standardization has been completed.

[1297] The "means for acquiring and recording the progress status at regular intervals" is a system function in which the server periodically monitors the progress of the inspection work and records the information.

[1298] The "means for displaying the acquired progress status in real time" is an interface that displays the recorded progress information so that the user can monitor it in real time.

[1299] The "means for updating and managing recorded progress status information" is a system function that appropriately updates progress information and corrects or supplements it as necessary.

[1300] The "means for generating a time series graph based on cleansed and standardized test data" is a function for creating a graph that visually represents data fluctuations based on processed test data.

[1301] The "means for saving the graph as an image file" is a function for saving the generated time series graph in image format.

[1302] "Means for predicting disease names from test results using a pre-trained predictive model" is a function that predicts diseases from new test data using a model trained on past data.

[1303] The "means for adding and recording prediction results to test data" is a function for adding the predicted disease name to existing test data and resaving it.

[1304] This invention is a system for automatically processing test data in the medical field, automating progress management, visualizing and analyzing data, and predicting the likelihood of test results. This system aims to improve the work efficiency of medical professionals by analyzing and managing test data uploaded by users on a server.

[1305] System Configuration

[1306] This system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data and running predictive models, while the terminal provides an interface for users to access and operate the system. Users primarily upload test data, check progress, and view analysis results.

[1307] Hardware and software used

[1308] The server is a computer system equipped with a powerful processor and sufficient memory, running a Linux-based OS and Apache HTTP Server, and uses an SQL-based database (e.g., MySQL) to store data.

[1309] The terminal is a PC or tablet operated by the user, and the system is accessed through a web browser such as Chrome or Firefox. The web interface is developed using HTML, CSS, and JavaScript.

[1310] Explaining program processing in natural language

[1311] 1. User uploads inspection data

[1312] Through the system interface, users upload test data CSV files to the server, which then stores the files in a specified directory.

[1313] 2. The server performs data cleansing and standardization

[1314] The server reads the uploaded test data, deletes incomplete data, standardizes the data, and converts it based on the calculated mean and standard deviation. The standardized data is then stored on a recording medium.

[1315] 3. The server periodically acquires and records the progress

[1316] The server obtains the inspection progress status at regular intervals and records the information in a text file, thereby always managing the latest progress status.

[1317] 4. The server visualizes the data

[1318] The server generates a time series graph based on the cleansed and standardized test data, which is then saved as an image file and made available to users via their devices.

[1319] 5. The server performs disease prediction

[1320] The server uses a pre-trained prediction model to predict the name of the disease from the test data. The prediction results are added to the test data and saved as a new CSV file. Users can view these prediction results on their devices.

[1321] Specific examples

[1322] Example 1: A concrete example of automated processing of test data

[1323] The user uploads a file named exam_data.csv to the system. The server saves this file in a specified directory and performs data cleansing and standardization. The cleansed data is saved as cleaned_exam_data.csv.

[1324] Example 2: Specific example of automated progress management

[1325] The server retrieves the current test progress every 60 seconds and records the information in progress_status.txt, allowing medical personnel to always be kept up to date with the latest progress.

[1326] Example 3: A concrete example of data visualization and analysis

[1327] The user uses the system interface to view a graph of the cleansed and standardized data, and the server generates a graph based on this data and saves it as exam_data_visualization.png.

[1328] Example 4: Specific example of predicting the likelihood of test results

[1329] When a user inputs test data into the system for diagnostic purposes, the server uses the trained prediction model to predict diseases based on the test results. The prediction results are added to the data and saved as exam_data_with_predictions.csv. Users can view the results and use them as a reference for diagnosis.

[1330] Prompt Sentence Examples

[1331] Upload the following test data for cleansing, standardization, and disease prediction.

[1332] Filename: exam_data.csv

[1333] This prompt causes the system to automatically begin processing the test data and generate results.

[1334] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1335] System program processing flow

[1336] Processing Steps

[1337] Step 1:

[1338] The user uploads the test data.

[1339] The user opens a web browser on their device and accesses the system interface (HTML form). Using the file selection dialog on the interface, they select the CSV file of the examination data (e.g., exam_data.csv) and click the "Upload" button. The server receives this file and saves it in a specified directory (e.g., / data / uploads / ).

[1340] Input: exam_data.csv (file uploaded by user)

[1341] Output: exam_data.csv saved in the specified directory

[1342] Step 2:

[1343] The server performs data cleansing.

[1344] The server reads the CSV files stored in the specified directory, analyzes the data, and removes rows containing incomplete data or missing values. The cleansed data is temporarily stored in temporary variables in memory.

[1345] Input: exam_data.csv in the specified directory

[1346] Output: Cleansed data (in memory) with imperfect data removed

[1347] Step 3:

[1348] The server performs the normalization of the data.

[1349] The server calculates the mean and standard deviation for each numeric item based on the parsed data, then normalizes each data point using these statistics. The normalized data is saved as a new CSV file (e.g., cleaned_exam_data.csv).

[1350] Input: Cleansed data (in memory)

[1351] Output: cleaned_exam_data.csv containing the standardized data

[1352] Step 4:

[1353] The server periodically obtains and records the progress.

[1354] The server checks the inspection progress every 60 seconds and records the progress data in a text file (e.g., progress_status.txt). This information includes the current data processing status and whether or not there were any errors.

[1355] Input: Internal server progress data

[1356] Output: progress_status.txt (latest progress information)

[1357] Step 5:

[1358] The server visualizes the data.

[1359] The server generates a time series graph based on the standardized data. This graph is saved in image format (e.g., PNG format). The graph is generated using the Python matplotlib library.

[1360] Input: cleaned_exam_data.csv (standardized data)

[1361] Output: exam_data_visualization.png (generated graph)

[1362] Step 6:

[1363] The server performs the disease prediction.

[1364] The server loads a pre-trained machine learning model (e.g., Scikit-learn's random forest model). Standardized test data is input into this model to predict the likelihood of disease. The prediction results are appended to the original data and saved as a new CSV file (e.g., exam_data_with_predictions.csv).

[1365] Input: Standardized test data, trained prediction model

[1366] Output: exam_data_with_predictions.csv (data with prediction results)

[1367] Step 7:

[1368] The user views the results.

[1369] Users can access the system interface through their terminal and download or view the prediction result file (e.g., exam_data_with_predictions.csv), allowing them to check the prediction results and use them as a reference for diagnosis.

[1370] Input: exam_data_with_predictions.csv (data with prediction results)

[1371] Output: Prediction result data viewed by the user

[1372] (Application example 1)

[1373] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1374] In modern factories, progress management and anomaly detection in production processes are important issues. However, there are insufficient means to efficiently carry out these management and detection tasks, which requires a lot of time and effort. In addition, it is difficult to quickly notify and respond when an anomaly occurs, which often leads to a decline in productivity and quality. Therefore, there is a need for efficient real-time progress management, anomaly detection, and data visualization.

[1375] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1376] In this invention, the server includes means for converting the inspection data into an electronic format and saving it, means for cleansing the inspection data to remove incomplete data, means for standardizing the cleansed inspection data, means for saving the standardized data on a recording medium, means for acquiring and recording a progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for detecting the possibility of an anomaly occurring using an anomaly detection model, means for notifying a detected anomaly via a smartphone, and means for displaying the data as a graph and saving it as an image file. This enables efficient management of the progress status of the production process, enables rapid response when an anomaly occurs, and is expected to improve productivity and quality.

[1377] "Inspection data" is information that includes measurements and status information related to a factory or manufacturing process.

[1378] "Cleansing" is the process of removing missing or outliers from data and shaping the data.

[1379] "Standardization" is the process of reducing variability in data and converting it to a uniform scale.

[1380] A "recording medium" is a physical or electronic means for storing data.

[1381] "Progress" is information that indicates the progress of work in a factory or manufacturing process.

[1382] "Real-time" means that the latest information is always acquired and displayed in real time.

[1383] "Update management" is the management process for keeping information and data up to date.

[1384] An "anomaly detection model" is a machine learning model for detecting abnormal values ​​in data.

[1385] "Anomaly" refers to a condition or occurrence that is different from the norm, including problems in the manufacturing process.

[1386] A "smartphone" is a mobile device that can connect to the Internet and use multifunctional applications.

[1387] "Graphical display of data" is a method of converting and displaying data in a form that is visually easy to understand.

[1388] An "image file" is a digital file format for storing visual information.

[1389] A "trained predictive model" is a model that has been trained using past data and can predict future states.

[1390] "Progress delay" is a phenomenon that indicates a delay between planned progress and actual progress.

[1391] "Production progress" is information indicating the current progress of work in the manufacturing process.

[1392] The system for realizing this invention is mainly composed of three elements: a server, a terminal, and a user. Each specific processing step will now be described.

[1393] 1. Upload your data

[1394] Users upload data generated by factory robots and manufacturing machines to a server in CSV file format using a smartphone interface.

[1395] 2. Data Cleansing and Standardization

[1396] The server receives the uploaded data, removes incomplete data, reduces the data variability, and converts it to a standard scale.

[1397] 3. Data storage

[1398] The cleansed and standardized data is stored on a storage medium, which is typically a database (e.g., MySQL or PostgreSQL).

[1399] 4. Automated progress management

[1400] The server collects and records the factory progress at regular intervals, and this information is updated in real time and can be viewed by users via a smartphone app.

[1401] 5. Anomaly detection and notification

[1402] The server analyzes the data using a pre-trained anomaly detection model (using TensorFlow or PyTorch) to detect anomalies, and notifies the user of any detected anomalies via their smartphone.

[1403] 6. Data visualization and analysis

[1404] The server generates a time series graph based on the cleansed and standardized data, which is saved as an image file (PNG file) and displayed to the user via a smartphone app.

[1405] 7. Progress forecast using predictive models

[1406] The server uses the trained prediction model to predict possible delays in progress or abnormalities. The prediction results are added to the data and saved again. This allows users to visually check the actual progress and take any necessary measures quickly.

[1407] Specific examples

[1408] For example, if a production line in a factory is behind schedule or if there is a possibility of a machine malfunction, the system will automatically visualize the progress and potential malfunction and warn the user, who can then check the progress in real time via a smartphone app.

[1409] Prompt Sentence Examples

[1410] Examples of prompts for generative AI models include:

[1411] Analyze factory production line data and predict the possibility of anomalies. Visualize progress taking into account production speed and machine status, and generate alerts if delays or anomalies are detected.

[1412] This will enable efficient management of factory production processes and improve productivity and quality.

[1413] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1414] Step 1:

[1415] The user uploads data generated by factory robots and manufacturing machines to the server in CSV file format. Specifically, the user uses a smartphone app to select the desired CSV file from the file selection screen. This input data includes numerical values ​​and status information about the manufacturing process. The server saves the received file in a temporary directory.

[1416] Step 2:

[1417] The server reads the uploaded CSV file and cleanses the data, specifically removing incomplete data and converting it to the required format. The input at this stage is the CSV file and the output is a cleansed DataFrame. The software used is Python and the Pandas library.

[1418] Step 3:

[1419] The server standardizes the cleansed data. Specifically, it calculates the mean and standard deviation of each data and standardizes the data. This standardized data is stored in a database. The input is a cleansed data frame, and the output is a standardized data frame. The software used is Python and the Scikit-learn library.

[1420] Step 4:

[1421] The server obtains the progress status at regular intervals and records that information. Specifically, the server saves the progress data it obtains from each manufacturing machine by appending it to a text file. The input is the progress data, and the output is a text file containing the progress status. This process is performed periodically using scheduling software.

[1422] Step 5:

[1423] The server displays the acquired progress status in real time. Specifically, it updates and displays the progress status information in real time on a web interface or smartphone app. The input is the progress status data, and the output is the latest progress status displayed on the user interface.

[1424] Step 6:

[1425] The server uses an anomaly detection model to detect possible anomalies in the data. Specifically, it inputs the cleansed and standardized data into the AI ​​model and calculates an anomaly score. The input is the standardized data, and the output is the anomaly score. The software used is TensorFlow or PyTorch.

[1426] Step 7:

[1427] The server notifies the user of detected anomalies via their smartphone. Specifically, if the anomaly score exceeds a threshold, it sends a push notification to the smartphone app. The input is the anomaly score, and the output is a notification message to the user.

[1428] Step 8:

[1429] The server generates a time series graph based on the cleansed and standardized data. Specifically, it generates the graph using Matplotlib or Plotly and saves it as an image file. The input is the standardized data, and the output is a time series graph image.

[1430] Step 9:

[1431] The server uses a trained prediction model to predict the possibility of progress delays or abnormalities. Specifically, it inputs standardized data into the prediction model and obtains a prediction result. The input is the standardized data, and the output is the prediction result. The software used is TensorFlow and PyTorch.

[1432] Step 10:

[1433] The server adds the prediction results to the data and saves them back to the database. Specifically, the obtained prediction results are added to the existing data frame and saved as a new one. The input is the prediction results and the existing data frame, and the output is the updated data frame.

[1434] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1435] This invention is a system that combines automatic processing of medical test data, automated progress management, data visualization and analysis, and probable test result prediction in the medical field with an emotion engine that recognizes the user's emotions. This system uses the emotion engine to monitor the user's emotions and provides appropriate interfaces and feedback according to the user's emotional state, thereby improving the work efficiency of medical professionals and the patient experience.

[1436] System Configuration

[1437] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running predictive models, and operating the emotion engine, while the terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[1438] Introducing the Emotion Engine

[1439] User Emotion Recognition

[1440] The server runs an emotion engine based on the user's voice, facial expressions, text input, etc. to recognize the user's emotional state, and provides appropriate feedback according to the user's emotional state.

[1441] Dynamically changing the display of test data based on emotions

[1442] The server dynamically changes how the test data is displayed based on the user's perceived emotions, for example, by displaying more detailed information and explanations if the user is anxious, to provide a sense of security.

[1443] Emotional state monitoring and collaboration

[1444] The emotion engine continuously monitors the user's emotional state, and the results are recorded in relation to progress, allowing changes in the user's mental state to be tracked.

[1445] Specific examples

[1446] Example 1: Emotion Recognition

[1447] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Based on this information, the server displays an interface that matches the user's mood.

[1448] Example 2: Dynamic data display

[1449] When the user tries to check the test results, the server recognizes that the user is anxious using its emotion engine. In addition to the usual display method, the server displays detailed explanations of the results and reference information to alleviate the user's anxiety.

[1450] Example 3: A concrete example of linking and monitoring emotional states

[1451] The server simultaneously runs an emotion engine to monitor the user's emotional state while the user checks the test data and views the progress. The results are recorded along with the progress so that medical professionals can later understand the user's emotional changes.

[1452] This system not only improves the work efficiency of medical professionals, but also provides services that take into consideration the mental health of patients. By combining it with an emotion engine, it becomes possible to provide responses that are optimized for each individual user, resulting in the provision of higher quality medical services.

[1453] The processing flow will be explained below.

[1454] Embodiment of a system combining emotion engines

[1455] Recognizing user emotions with an emotion engine

[1456] Step 1:

[1457] The user logs into the system.

[1458] The server receives the user's login information and performs authentication.

[1459] Step 2:

[1460] When the user initiates an action, the device's camera and microphone become active.

[1461] The device collects the user's facial expressions and voice data in real time.

[1462] Step 3:

[1463] The server sends the collected facial expression and voice data to the emotion engine.

[1464] The emotion engine uses facial expression analysis and voice analysis algorithms to determine the user's emotional state.

[1465] Step 4:

[1466] The emotion engine determines the user's emotion and returns the result to the server.

[1467] The server receives the data and records the user's current emotional state.

[1468] Dynamically changing the display of test data based on emotions

[1469] Step 1:

[1470] The user performs operations to view the inspection data.

[1471] The terminal sends the request to the server.

[1472] Step 2:

[1473] The server ascertains the user's emotional state.

[1474] Refer to the data of the emotional state determined by the emotion engine.

[1475] Step 3:

[1476] The server changes how the test data is displayed based on the user's emotional state.

[1477] For example, if the user is in an anxious state, detailed explanations or reassuring messages are displayed.

[1478] Step 4:

[1479] The display content is transmitted to the terminal based on the changed display method.

[1480] The terminal displays the test data in a user-readable format.

[1481] Emotional state monitoring and collaboration

[1482] Step 1:

[1483] While the user is using the system, the device continuously collects facial expression and voice data.

[1484] The collected data is periodically sent to a server.

[1485] Step 2:

[1486] The server periodically sends data to the emotion engine to update the user's emotional state.

[1487] The updated emotion data is saved to the server in real time.

[1488] Step 3:

[1489] The server integrates the progress status data and the emotion data, and records the changes in the user's emotional state in association with the progress status.

[1490] The recorded data is later used by medical personnel to ascertain the user's emotional state.

[1491] Step 4:

[1492] When the user exits the system, the server records the final emotional state and ends the session.

[1493] The device will disable the camera and microphone and stop transmitting collected data.

[1494] <<Example>>

[1495] Example 1: User Emotion Recognition

[1496] When a user logs in to the system, the device's camera and microphone collect facial expressions and voice in real time. The server sends this data to an emotion engine to recognize when the user is feeling stressed. Based on the results, the server displays an interface using a theme color that has a relaxing effect on the user.

[1497] Example 2: Dynamic data display based on emotions

[1498] When a user attempts to view a particular test result, the server detects through an emotion engine that the user is feeling anxious, and the device displays the test data with detailed explanations and a message saying, "If you have any questions, please consult your doctor."

[1499] Example 3: Monitoring and recording emotional states

[1500] While the user is checking their progress, the server uses an emotion engine to monitor their emotional state and records the results along with their progress. If the user becomes frustrated, the server notifies medical professionals and prompts them to take appropriate action.

[1501] These steps will enable the system to achieve more flexible and efficient medical data management and provision that takes into account users' emotions.

[1502] Example 2

[1503] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1504] The modern medical field requires efficient processing and management of test data. However, existing systems are unable to adequately address the following: test data cleansing, standardization, storage, real-time display and recording of progress, data visualization, and disease prediction using predictive models. They also lack the ability to recognize the user's emotional state and provide appropriate feedback and interfaces accordingly. This makes it difficult to improve the work efficiency of medical professionals and the psychological satisfaction of patients.

[1505] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for analyzing the user's voice, facial expression, and text input to recognize the user's emotional state, means for dynamically changing the display method of the test data based on the recognized emotional state, and means for continuously monitoring the user's emotional state and recording the results in association with the progress status. This enables efficient processing and management of test data and further enables the provision of appropriate feedback and interfaces according to the user's emotional state. This improves the work efficiency of medical professionals and increases the psychological satisfaction of patients.

[1506] "Test data" refers to information including results and measurements obtained by a user at a medical institution or testing facility.

[1507] "Cleansing" is the process of removing incomplete data and errors from test data and organizing it.

[1508] "Standardization" is the process of aligning cleansed data into a consistent format and units.

[1509] A "recording medium" is a digital storage device or service for storing data.

[1510] "Progress" is information that indicates the progress and current status of medical treatments and examinations.

[1511] "Emotional state" refers to the user's psychological state or mood.

[1512] "Voice analysis" is a technology that analyzes voice data and extracts specific patterns and emotions.

[1513] "Facial expression analysis" is a technology that analyzes facial expressions captured by a camera and recognizes emotions.

[1514] "Text analysis" is the process of analyzing textual information entered by a user to understand their emotions and intent.

[1515] "Dynamic change" is a mechanism that changes the display content and behavior of the system in real time according to specific conditions or situations.

[1516] "Monitoring" is the act of continuously observing a user's state and behavior and collecting data.

[1517] "Feedback" is information or a response provided to a user with the purpose of approving or correcting their behavior.

[1518] This system is specialized for efficient processing and management of medical test data. It automates a series of processes, from test data cleansing to standardization, storage, and sentiment analysis, with the aim of improving the work efficiency of medical professionals and the psychological satisfaction of patients.

[1519] System Configuration

[1520] The system consists of three components: a server, a terminal, and a user. The server processes and stores data, runs predictive models, and operates the emotion engine. The terminal provides an interface for users to access and operate the system. Users upload test data, check progress, and view analysis results and emotion-based feedback.

[1521] Hardware and software used

[1522] Hardware: Use a server equipped with a high-performance processor and a large amount of memory (e.g., Dell PowerEdge, HP ProLiant) as the server. Use a personal computer or tablet equipped with a camera and microphone as the terminal.

[1523] Software: The server uses "EmotionAPI," "database management system (e.g., MySQL, PostgreSQL)," and "web server (e.g., Apache, Nginx)." The terminal uses technologies such as "WebRTC" and "JavaScript framework (e.g., React.js)."

[1524] Data processing and calculation

[1525] 1. Cleansing: The server receives the uploaded test data and filters out missing or inappropriate data.

[1526] 2. Standardization: The cleansed data is converted into a unified format and stored in a database.

[1527] 3. Emotion recognition: The user's voice, facial expressions, and text input sent from the device are analyzed using the "Emotion API" to recognize the user's emotional state.

[1528] 4. Dynamic Change: The server dynamically changes how the test data is displayed based on the perceived emotional state. For example, if the user is anxious, it provides more information and reassuring feedback.

[1529] 5. Monitoring: The server continuously monitors the user's emotional state and records it in relation to their progress.

[1530] Specific examples

[1531] Example 1: Emotion Recognition

[1532] When a user logs in to the system, the device uses a camera and microphone to capture the user's facial expressions and voice. The server analyzes the data using the "Emotion API" to recognize the user's emotional state. If the user is feeling anxious, the server will provide an interface that provides a sense of security.

[1533] Example 2: Dynamic data display

[1534] When a user attempts to check the results of their test data, the device sends a request to the server. The server recognizes that the user is anxious and sends the results to the device with a detailed explanation and reference information. The device then displays the results to the user.

[1535] Example 3: A concrete example of emotional state monitoring

[1536] The server uses an emotion engine to monitor the user's emotional state while they are reviewing the test data and viewing their progress, and records the results in association with their progress so that medical professionals can review them later.

[1537] Prompt Sentence Examples

[1538] When a user logs in to the system, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Then, after reviewing the test data, the server displays the results with detailed explanations and provides feedback to the user to give them a sense of security.

[1539] This allows the system to not only efficiently process and manage test data, but also to provide an interface and feedback that takes user emotions into consideration, thereby improving the work efficiency of medical professionals and increasing the psychological satisfaction of patients.

[1540] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1541] Step 1:

[1542] When a user logs into the system, the terminal takes the user's email address and password as input and sends them to the server. The server queries a database to verify the authentication information. If authentication is successful, the server outputs a login success message to the terminal. The terminal receives this and notifies the user that the login was successful.

[1543] Step 2:

[1544] The device uses a camera and microphone to capture the user's facial expressions and voice. This captured data is sent as input to the server. The server analyzes the received data using the Emotion API to recognize the user's emotional state. Based on this result, the server generates appropriate user interface data and outputs it to the device. The device then displays the received interface data.

[1545] Step 3:

[1546] Users upload test data files using a drag-and-drop interface on their devices, which then send the files as input to the server, which cleanses the received test data and removes incomplete data. The cleansed data is then standardized and stored in a database.

[1547] Step 4:

[1548] The user requests confirmation of test data from the device. The device sends this request as input to the server. The server retrieves the relevant test data from the database and again uses the Emotion API to recognize the user's current emotional state. The server receives the test data and emotional state as input and generates detailed feedback. The device displays this feedback.

[1549] Step 5:

[1550] The device continuously captures the user's voice and facial expressions using a camera and microphone. This periodically captured data is sent as input to a server. The server uses the Emotion API to recognize the user's emotional state and record it in a database. The results are associated with progress and stored in the database. Medical professionals can access the database to check changes in the user's emotional state.

[1551] (Application example 2)

[1552] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1553] Conventional food delivery applications provide menu recommendations and feedback without considering the user's emotional state. As a result, it is difficult to provide appropriate support to users who are feeling stressed or anxious, limiting the quality of the user experience. Furthermore, they are unable to provide a dynamic interface based on the user's emotions, and improvements that would lead to increased user satisfaction are needed.

[1554] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting test data into an electronic format and saving it, means for cleansing the test data to remove incomplete data, means for standardizing the cleansed test data, means for saving the standardized data on a recording medium, means for acquiring and recording progress status at regular time intervals, means for displaying the acquired progress status in real time, means for updating and managing the recorded progress status information, means for providing appropriate feedback based on the user's emotions using an emotion engine that recognizes the user's emotional state by analyzing the user's facial expressions and voice, and means for analyzing the user's facial expressions and presenting dietary recommendations based on the analysis. This enables feedback and recommendations that take the user's emotional state into consideration, significantly improving the quality of the user experience.

[1555] "Electronic test data format" refers to the technical means for reading test data, converting the data into a digital format, and storing it electronically.

[1556] "Cleansing" is the process of removing incomplete data and noise from inspection data to improve data quality.

[1557] "Standardization" is a technical means of converting cleansed data into a uniform format that makes it easier to compare and analyze.

[1558] "Storage medium" means a physical or electronic device or system for storing standardized data.

[1559] "Status capture" is the process of periodically reviewing and recording data and project progress.

[1560] "Real-time display" refers to a technical means that instantly reflects progress and data and displays them to the user.

[1561] Update management is the process of keeping recorded progress information up to date and managing changes and additions.

[1562] An "emotion engine" is an algorithm or technology that analyzes and recognizes a user's emotional state from their facial expressions and voice.

[1563] "Feedback" is a response that provides appropriate information or recommendations depending on the user's perceived emotional state.

[1564] "Dietary recommendations" are information that suggests meals and menus that are optimal for the user's emotional state at the time, based on the user's facial expression.

[1565] "Data visualization" is a technical method of displaying inspection data in the form of graphs and charts to make it easier to understand.

[1566] "Dynamic display change" is a technical means for changing the way data is displayed in real time according to the user's emotional state.

[1567] "Emotion monitoring" is the process of continuously tracking a user's emotional state and recording its changes.

[1568] A "predictive model" is an algorithm or technique that uses a pre-trained dataset to predict outcomes from new data.

[1569] A "prompt" is an instruction or command to be input into a generative AI model.

[1570] The present invention provides a system for recognizing a user's emotions in food delivery and providing appropriate feedback and recommendations based on the emotions. Specific embodiments of the system are described below.

[1571] System Configuration

[1572] The system consists of three components: a server, a terminal, and a user. The server is primarily responsible for processing and storing data, running the emotion engine, and operating the predictive model. The terminal provides an interface for users to access and operate the system. Users order food and check feedback.

[1573] Introducing the Emotion Engine

[1574] User Emotion Recognition

[1575] The server uses an image processing library (e.g., OpenCV) and an emotion recognition model (e.g., a model trained with Keras) to analyze the user's facial expressions. Images and videos uploaded by the user are taken, and the server processes them to recognize the user's emotional state.

[1576] Emotion-based dietary recommendations

[1577] The server provides appropriate feedback based on the recognized emotion, suggesting the most suitable menu from a predefined list to provide dietary recommendations according to the user's emotional state. For example, if the user is anxious, it will recommend relaxing meals and herbal teas.

[1578] Emotional state monitoring and collaboration

[1579] The server runs an emotion engine in real time to monitor the user's emotional state when confirming an order, records the results sequentially, and analyzes long-term trends to provide strategies to improve user satisfaction.

[1580] Specific examples

[1581] Examples of facial expression analysis and emotion recognition

[1582] When a user accesses the system, they take a picture of their face using the device's camera and upload it to the server. The server processes the image and recognizes the user's emotional state. Specific examples of prompts are as follows:

[1583] Analyze the image file "user_face.jpg" and recognize the user's emotions.

[1584] Provide appropriate feedback based on predicted emotions.

[1585] Dynamic Data Display and Feedback Example

[1586] When the user tries to confirm their order, the server analyzes the user's emotions and dynamically changes the feedback based on the results. For example, if the user is anxious, a recommendation message such as "Try a menu that has a relaxing effect" will be displayed.

[1587] Hardware and software used

[1588] Hardware: The user's device (such as a smartphone or tablet)

[1589] Software: Image processing libraries (e.g., OpenCV), machine learning frameworks (e.g., Keras), frameworks (e.g., Flask) on the server

[1590] In this way, personalized responses that take into account the user's emotional state can be made, significantly improving the user experience.

[1591] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1592] Step 1:

[1593] A user takes a picture of their face using the device camera and uploads it to the system. The input is the user's face image file, and the output is the image data sent to the server.

[1594] Step 2:

[1595] The server analyzes the received facial image data using the OpenCV library. Specifically, it uses a facial recognition algorithm to detect the face and extract the necessary parts. The input is the user's facial image data, and the output is image data of the facial area.

[1596] Step 3:

[1597] The server inputs the facial image data into an emotion recognition model trained with Keras to predict emotions. The input is the facial image data, and the output is the user's emotional state (e.g., "anxiety," "joy," "anger," etc.).

[1598] Step 4:

[1599] The server selects appropriate feedback and dietary recommendations based on the predicted emotional state. The input is the emotional state, and the output is a feedback message or recommended menu. For example, it performs a specific action such as "if the user is anxious, recommend a relaxing herbal tea."

[1600] Step 5:

[1601] The server sends the selected feedback and recommended menu to the terminal. The input is the feedback message and recommended menu, and the output is the information displayed on the terminal.

[1602] Step 6:

[1603] The terminal displays the feedback messages and recommended menus received from the server to the user. The input is the feedback information from the server, and the output is the information visually displayed to the user.

[1604] Step 7:

[1605] The server records the user's order history and feedback, and monitors changes in their emotional state. The input is the user's order and feedback data, and the output is continuous monitoring data. This allows us to analyze long-term user satisfaction and use it to further improve our services.

[1606] In this way, each processing step is performed sequentially, providing real-time feedback and recommendations to improve the user experience.

[1607] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1608] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1609] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1610] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1611] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1612] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1613] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1614] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1615] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1616] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1617] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1618] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1619] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1620] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1621] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1622] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1623] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1624] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1625] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1626] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1627] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1628] The following is further disclosed regarding the above embodiment.

[1629] (Claim 1)

[1630] means for converting and storing the test data in an electronic format;

[1631] a means for cleansing the test data to remove incomplete data;

[1632] a means for standardizing the cleansed laboratory data;

[1633] means for storing the standardized data on a recording medium;

[1634] a means for capturing and recording progress at regular time intervals;

[1635] a means for displaying the obtained progress in real time;

[1636] The system includes a means for updating and managing the recorded progress information.

[1637] (Claim 2)

[1638] A means for visualizing the inspection data and displaying it as a graph;

[1639] 10. The system of claim 1, further comprising means for saving the graph as an image file.

[1640] (Claim 3)

[1641] A means for predicting the name of a disease from test results using a trained prediction model;

[1642] 2. The system according to claim 1, further comprising means for recording the prediction result together with the test data.

[1643] "Example 1"

[1644] (Claim 1)

[1645] means for converting and storing the test data in an electronic format;

[1646] a means for a user to upload test data;

[1647] means for saving the test data to a designated directory;

[1648] a means for cleansing the test data to remove incomplete data;

[1649] a means for standardizing the cleansed laboratory data;

[1650] means for storing the standardized data on a recording medium;

[1651] a means for capturing and recording progress at regular time intervals;

[1652] a means for displaying the obtained progress in real time;

[1653] The system includes a means for updating and managing the recorded progress information.

[1654] (Claim 2)

[1655] a means for generating a time series graph based on the cleansed and standardized test data;

[1656] 10. The system of claim 1, further comprising means for saving the graph as an image file.

[1657] (Claim 3)

[1658] A means of predicting the name of a disease from test results using a pre-trained prediction model;

[1659] 2. The system according to claim 1, further comprising means for recording the prediction result together with the test data.

[1660] "Application Example 1"

[1661] (Claim 1)

[1662] means for converting and storing the test data in an electronic format;

[1663] a means for cleansing the test data to remove incomplete data;

[1664] a means for standardizing the cleansed laboratory data;

[1665] means for storing the standardized data on a recording medium;

[1666] a means for capturing and recording progress at regular time intervals;

[1667] a means for displaying the obtained progress in real time;

[1668] A means for updating and managing the recorded progress information;

[1669] a means for detecting a possible occurrence of an anomaly using an anomaly detection model;

[1670] A means for notifying detected abnormalities via a smartphone;

[1671] A means to display the data as a graph and save it as an image file;

[1672] A system including:

[1673] (Claim 2)

[1674] A means for visualizing the inspection data and displaying it as a graph;

[1675] 10. The system of claim 1, further comprising means for saving the graph as an image file.

[1676] (Claim 3)

[1677] A means for predicting progress delays and production progress using a trained prediction model;

[1678] 2. The system according to claim 1, further comprising means for recording the prediction result together with the test data.

[1679] "Example 2: Combining Emotion Engines"

[1680] (Claim 1)

[1681] means for converting and storing the test data in an electronic format;

[1682] a means for cleansing the test data to remove incomplete data;

[1683] a means for standardizing the cleansed laboratory data;

[1684] means for storing the standardized data on a recording medium;

[1685] a means for capturing and recording progress at regular time intervals;

[1686] a means for displaying the obtained progress in real time;

[1687] A means for updating and managing the recorded progress information;

[1688] means for analyzing a user's voice, facial expression, and text input to recognize their emotional state;

[1689] means for dynamically altering the display of test data based on the recognized emotional state;

[1690] A system including a means for continuously monitoring a user's emotional state and recording the results in relation to progress.

[1691] (Claim 2)

[1692] A means for visualizing the inspection data and displaying it as a graph;

[1693] A means to save the graph as an image file;

[1694] 10. The system of claim 1, further comprising means for dynamically modifying the user interface based on an emotional state.

[1695] (Claim 3)

[1696] A means for making predictions from test results using a trained prediction model;

[1697] a means for adding the prediction result to the inspection data and recording it;

[1698] 10. The system of claim 1, further comprising means for recording changes in the user's emotional state for review by a medical professional.

[1699] "Application example 2 when combining emotion engines"

[1700] (Claim 1)

[1701] means for converting and storing the test data in an electronic format;

[1702] a means for cleansing the test data to remove incomplete data;

[1703] a means for standardizing the cleansed laboratory data;

[1704] means for storing the standardized data on a recording medium;

[1705] a means for capturing and recording progress at regular time intervals;

[1706] a means for displaying the obtained progress in real time;

[1707] A means for updating and managing the recorded progress information;

[1708] a means for providing appropriate feedback based on the user's emotions using an emotion engine that recognizes the user's emotional state by analyzing the user's facial expressions and voice;

[1709] means for analyzing a user's facial expressions and providing dietary recommendations based thereon;

[1710] A system including:

[1711] (Claim 2)

[1712] A means for visualizing the inspection data and displaying it as a graph;

[1713] A means to save the graph as an image file;

[1714] 10. The system of claim 1, further comprising means for dynamically changing the way the graph is displayed in response to the emotional state of the user.

[1715] (Claim 3)

[1716] A means for predicting the name of a disease from test results using a trained prediction model;

[1717] a means for adding the prediction result to the inspection data and recording it;

[1718] 10. The system of claim 1, further comprising means for continuously monitoring the user's emotional state based on the emotion engine and recording the results in association with the progress. [Explanation of symbols]

[1719] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for converting and storing the test data in an electronic format; a means for cleansing the test data to remove incomplete data; a means for standardizing the cleansed laboratory data; means for storing the standardized data on a recording medium; a means for capturing and recording progress at regular time intervals; a means for displaying the obtained progress in real time; The system includes a means for updating and managing the recorded progress information.

2. A means for visualizing the inspection data and displaying it as a graph; 10. The system of claim 1, further comprising means for saving the graph as an image file.

3. A means for predicting the name of a disease from test results using a trained prediction model; 2. The system according to claim 1, further comprising means for recording the prediction result together with the inspection data.

Images