System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting real-time bio-data of animals from sensing devices; means for receiving video and analyzing behavior patterns of the animals; means for analyzing the bio-data and the video by a generation AI and detecting changes in health conditions and behavior patterns of the animals; means for storing analysis results in a database and generating an alert when an abnormality is detected; and means for notifying a user of the analysis results and alert information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, monitoring of animal health and behavioral patterns has been primarily done manually. However, this method is limited in the information that can be collected, making it difficult to respond quickly. Furthermore, even in some facilities that use sensing devices and video analysis tools, there are limitations to the analysis and application of the data. Therefore, there is a need for a system that can more efficiently and comprehensively monitor animal health and behavioral patterns, detect abnormalities early, and take appropriate action. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting animal ecological data in real time from a sensing device, a means for receiving video data and analyzing the animal's behavioral patterns, a means for analyzing the ecological data and video data using a generation AI to detect changes in the animal's health condition and behavioral patterns, a means for saving the analysis results in a database and generating an alert when an abnormality is detected, and a means for notifying the user of the analysis results and alert information. This system enables accurate and efficient monitoring of the animal's health condition and behavioral patterns, enabling prompt response.

[0006] A "sensing device" is a device that measures an animal's physical condition (heart rate, body temperature, location information, etc.) in real time and collects the data.

[0007] "Video data" refers to video information that records the behavior of animals obtained from a video acquisition device such as a camera.

[0008] A "behavioral pattern" refers to a sequence of specific behaviors (e.g., walking, resting, foraging, etc.) performed by an animal over a period of time.

[0009] "Generative AI" is a technology that uses artificial intelligence to analyze collected data and detect specific patterns and anomalies.

[0010] "Real-time" refers to the ability to process, analyze, and display data instantly, without delay, so that responses can be made almost simultaneously with actual time.

[0011] "Analysis results" refers to information obtained after analyzing data obtained from sensing devices or video data using artificial intelligence or other methods.

[0012] A "database" is a collection of data that is systematically organized and stored so that it can be easily searched and used later.

[0013] An "alert" is a warning message or signal that is sent when an abnormality is detected.

[0014] "User" refers to any person or organization that uses this system to monitor and manage the health and behavioral patterns of animals.

[0015] A "dashboard" is an information display screen that allows users to visually check analysis results and alert information. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] System Overview

[0038] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[0039] Program Description

[0040] Server Processing

[0041] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[0042] 1. Data Collection

[0043] The server receives real-time biological data such as heart rate, body temperature, and location information from the sensing devices attached to the animals, allowing the server to instantly grasp the animal's condition.

[0044] The server receives video data from cameras installed in the breeding area and captures the behavior of the animals.

[0045] 2. Data Analysis

[0046] The collected ecological and video data is analyzed by generative AI, which uses this data to analyze the animals' health and behavioral patterns and detect any abnormalities.

[0047] For example, if the AI ​​detects an abnormally elevated heart rate, it will determine whether this is a sign of stress or illness.

[0048] 3. Data Retention and Notification

[0049] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[0050] If an abnormality is detected, the server immediately generates an alert and notifies the user. Alerts are classified according to the type and urgency of the abnormality.

[0051] Terminal handling

[0052] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[0053] 1. Receiving and displaying data

[0054] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[0055] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[0056] 2. Managing Notifications

[0057] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[0058] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[0059] User Action

[0060] Users use the system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[0061] 1. Data Monitoring

[0062] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[0063] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[0064] 2. Data Management and Feedback

[0065] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[0066] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[0067] Specific examples

[0068] A specific example is a health management system for lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action.

[0069] This system allows for efficient monitoring of animal health, early detection of abnormalities, and rapid response, thereby improving animal welfare and optimizing the breeding environment.

[0070] The processing flow will be explained below.

[0071] Server Processing

[0072] Step 1: Data collection

[0073] server:

[0074] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[0075] The server receives video data from cameras installed in the breeding area.

[0076] The collected data is stored in a temporary storage area.

[0077] Step 2: Preprocessing the data

[0078] server:

[0079] The server cleans the collected biological data, correcting and deleting inaccurate or missing data.

[0080] Noise removal and frame correction are also performed on the video data.

[0081] Step 3: Data analysis

[0082] server:

[0083] The server uses generative AI to analyze biological data in real time, detecting abnormalities in heart rate and body temperature.

[0084] The server uses video analysis tools to analyze the animal's behavioral patterns and detect abnormal behavior.

[0085] Step 4: Save Data

[0086] server:

[0087] The analysis results are stored in a database, which allows data to be accumulated over a long period of time.

[0088] The data will be organized chronologically to allow for quick searching and reference.

[0089] Step 5: Alerting and Notification

[0090] server:

[0091] If an anomaly is detected, the server immediately generates an alert, which includes the type of anomaly and its urgency.

[0092] Sends generated alerts to the device.

[0093] Terminal handling

[0094] Step 1: Receiving Data

[0095] Device:

[0096] The terminal receives analysis results and alert information from the server in real time.

[0097] Step 2: Visualize the data

[0098] Device:

[0099] Generate graphs and charts to visually display the received data, such as changes in heart rate or body temperature.

[0100] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[0101] Step 3: Sending notifications

[0102] Device:

[0103] Alert information is notified to the user via push notification, email, etc.

[0104] User Action

[0105] Step 1: Data monitoring

[0106] User:

[0107] Users can monitor analysis results and alert information in real time via their terminals.

[0108] If an important alert occurs, immediately check the notification and take the necessary action.

[0109] Step 2: Implementing countermeasures

[0110] User:

[0111] Based on the countermeasures presented by the server, appropriate care is provided for the animal's health condition and behavioral patterns.

[0112] For example, if an abnormality is detected, a veterinarian will be contacted and a detailed health check will be requested.

[0113] Step 3: Data feedback

[0114] User:

[0115] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[0116] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities in the future.

[0117] In this way, the entire process from collecting animal ecological data to analysis, notification, and response is carried out reliably.

[0118] Example 1

[0119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] Modern animal husbandry environments require effective systems for real-time monitoring of animal health and behavioral patterns and early detection of abnormalities. Conventional methods have fragmented processes for data collection, analysis, and notification, making it difficult to respond immediately. They also lack an interface that allows users to intuitively grasp the health status of animals. To solve these problems and improve animal welfare, an integrated system must be built.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0122] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for sending the generated alert to a terminal and the terminal notifying the user of it in real time, means for visually displaying the analysis results through a user interface and visualizing them as graphs or charts, and means for the user to refer to past data and analyze trends. This makes it possible to efficiently monitor the animal's health condition and behavioral patterns, detect abnormalities early, and take appropriate measures promptly.

[0123] A "sensing device" is a device that is attached to an animal and collects biological data such as heart rate, body temperature, and location information in real time.

[0124] "Video data" refers to digital data that records the movements and behavior of animals, captured using a video device such as a camera.

[0125] "Generative AI" is an artificial intelligence technology that analyzes ecological and video data to detect changes in animals' health and behavioral patterns.

[0126] A "database" is an information storage system for saving analysis results, and provides long-term data accumulation and backup functions.

[0127] An "alert" is a warning message that is generated when an abnormality is detected, and is a means of notifying the user.

[0128] A "terminal" is an interface device that allows users to use the system and receives and displays alerts and analysis results from the server.

[0129] A "user interface" is a collection of screens and functions that contain visual and operational elements that allow a user to interact with a system.

[0130] A "graph" is a diagram for visually displaying data, visualizing fluctuations along a time axis.

[0131] A "chart" is a graphic representation of numerical data visually, and includes formats such as bar graphs and line graphs.

[0132] A "trend" refers to a tendency or tendency observed from past data.

[0133] System Overview

[0134] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[0135] Program Description

[0136] Server Processing

[0137] The server is the center of this system, and performs data collection, analysis, storage, and notification. Specifically, it performs the following processes:

[0138] 1. Data Collection

[0139] The server receives real-time biological data, such as heart rate, body temperature, and location information, from sensing devices attached to the animals. For example, it receives data that a lion's heart rate has increased from 60 bpm to 80 bpm.

[0140] The server receives video data from cameras installed in the enclosure, capturing footage of the lions' movements and postures every 10 seconds.

[0141] 2. Data Transmission

[0142] The server simultaneously receives the collected biological data and video data, so it needs to maintain stable data communication over the network. For example, the server synchronizes data with the device using Wi-Fi.

[0143] 3. Data Analysis

[0144] The generative AI model on the server analyzes the received biological data and video data in real time. Specifically, the AI ​​detects abnormal heart rate patterns and behavioral abnormalities based on the video data. For example, it detects stress by linking a sudden increase in heart rate with vigorous movements in the video data.

[0145] 4. Data storage and alert generation

[0146] The server stores the results of the analysis by the generative AI in a database. For example, the analysis results are stored in a database as a lion's health log, including heart rate, body temperature, and behavioral patterns.

[0147] The server generates an alert if it detects an abnormality. For example, if the heart rate is abnormally high, the server adds "Abnormal Heart Rate: Urgent" to the alert list.

[0148] 5. User Notices

[0149] The server sends the generated alerts to the device. For example, when a new alert is generated, the server notifies the user by push notification or email.

[0150] The device notifies the user of the received alert in real time. For example, the device may use the smartphone's push notification function to send the user a message saying, "The lion's heart rate is abnormal. Please check immediately."

[0151] Terminal handling

[0152] The terminal is an interface that allows users to use the system. Specifically, it includes the following processes:

[0153] 1. Receiving and displaying data

[0154] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[0155] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[0156] 2. Managing Notifications

[0157] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[0158] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[0159] User Action

[0160] Users use the system to manage the health and behavioral patterns of their animals, including the following processes:

[0161] 1. Data Monitoring

[0162] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[0163] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[0164] 2. Data Management and Feedback

[0165] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[0166] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[0167] Specific examples

[0168] A specific example is a health management system for lions at a zoo. The server receives heart rate and body temperature data in real time from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action. This system makes it possible to efficiently monitor the health of animals, detect abnormalities early, and respond quickly. This improves animal welfare and optimizes the captive environment.

[0169] Prompt Sentence Examples

[0170] "Analyze the heart rate and temperature data received from the sensing device attached to the lion, as well as the camera footage, to assess the lion's health and behavioral patterns. If you detect any abnormalities, state the reason and recommend a course of action."

[0171] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0172] Step 1:

[0173] Data collection

[0174] The server receives biological data such as heart rate, body temperature, and location information in real time from sensing devices attached to the animals. The input is data from various sensors, and the output is raw data stored in the server. Specifically, the server receives and temporarily stores data on heart rate increases from 60 bpm to 80 bpm and body temperature data every unit of time.

[0175] Step 2:

[0176] Video data collection

[0177] The server receives video data from cameras installed in the enclosure. The input is the video signal from the camera, and the output is the video data stored in the server. For example, the server captures and stores video of lions' movements every 10 seconds.

[0178] Step 3:

[0179] Data transmission

[0180] The server communicates via a network to stably receive the collected biological data and video data. The input is data from the sensing device and camera, and the output is the integrated data sent to the server. Specifically, the server synchronizes the real-time data using Wi-Fi.

[0181] Step 4:

[0182] Data analysis

[0183] The generative AI model on the server analyzes the received ecological data and video data in real time. The input is the ecological data and video data stored on the server, and the output is the analyzed health status and behavioral pattern changes. Specifically, the AI ​​analyzes sudden increases in heart rate and violent animal movements based on the video to detect stress or health abnormalities.

[0184] Step 5:

[0185] Data storage

[0186] The server stores the results of the analysis by the generative AI in a database. The input is the analyzed health data and behavioral pattern data, and the output is the historical data stored in the database. Specifically, the server adds logs of the lion's heart rate, body temperature, and behavioral patterns to the database.

[0187] Step 6:

[0188] Alert Generation

[0189] The server generates an alert if it detects an abnormality. The input is the abnormal data from the generation AI, and the output is a warning message added to the alert list. For example, if the heart rate is abnormally high, the server adds "Heart rate abnormal: emergency" to the alert list.

[0190] Step 7:

[0191] Alert Notification

[0192] The server sends the generated alert to the device. The input is the alert message, and the output is the alert notification sent to the device. Specifically, the server sends an emergency message to the user via push notification or email.

[0193] Step 8:

[0194] Real-time notifications

[0195] The terminal notifies the user of the received alert in real time. The input is the alert notification sent from the server, and the output is a warning message displayed on the user's terminal. For example, the terminal may display a message such as "The lion's heart rate is abnormal. Please check immediately."

[0196] Step 9:

[0197] Data Monitoring

[0198] The user uses a terminal to monitor the analysis results and alert information from the server in real time. The input is the analysis results displayed on the terminal's dashboard, and the output is the user's judgment and response. Specifically, the user looks at the graphs and charts on the terminal to identify abnormalities.

[0199] Step 10:

[0200] Data Management and Feedback

[0201] Users can view long-term data to understand trends in animal health and behavior. The input is historical database information, and the output is analysis results and feedback. Specifically, users can view a year's worth of health data in graphs and provide feedback to the system to improve the accuracy of the generative AI.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] In recent years, as the complexity and operating hours of robots used in factories have increased, it has become increasingly important to monitor the status of robots in real time and detect abnormalities early. However, current systems lack a mechanism for immediately detecting and efficiently notifying robot abnormalities, which is a factor that affects production efficiency and safety. A solution to this issue is needed.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes means for collecting operational data of the object from the sensing device in real time, means for receiving video data and analyzing the work pattern of the object, means for analyzing the operational data and video data using a generation AI and detecting changes in the operating state and work pattern of the object, means for saving the analysis results in a database and generating an alert when an abnormality is detected, and means for notifying engineers of the analysis results and alert information. This makes it possible to monitor robot abnormalities in real time and to immediately notify engineers when an abnormality is detected.

[0207] A "sensing device" is a device that collects operational data and environmental data (temperature, vibration, power consumption, etc.) of an object in real time.

[0208] "Operation data" refers to data on the status of an object while it is in operation (such as temperature, vibration, and power consumption).

[0209] "Video data" refers to video information of an object being worked on, captured using photographic equipment such as a camera.

[0210] "Work pattern" refers to the pattern of actions and behaviors of an object when performing a specific task or operation.

[0211] "Generative AI" is artificial intelligence that analyzes changes in the operating status and work patterns of objects based on collected operational data and video data.

[0212] "Analysis results" refer to the information and insights obtained when operational data and video data are analyzed by the generating AI.

[0213] A "database" is a storage medium or system for long-term storage and management of analysis results and ecological data.

[0214] An "alert" is a warning notification that is generated when an abnormality is detected and is immediately reported to a technician.

[0215] An "engineer" is someone who receives analysis results and alert information and takes action.

[0216] A "dashboard" is a user interface that allows engineers to visually check analysis results and alert information.

[0217] System Overview

[0218] This invention is a system that enables preventive maintenance by monitoring the status and operation patterns of factory robots in real time and immediately notifying engineers if an abnormality is detected. The system mainly consists of a sensing device, a video analysis tool, a generative AI, and a user interface with a dashboard.

[0219] Program Description

[0220] Server Processing

[0221] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[0222] 1. Data Collection

[0223] The server receives real-time operational data such as temperature, vibration, and power consumption from sensing devices attached to factory robots, allowing the status of the target object to be grasped immediately.

[0224] The server receives video data from cameras installed in the factory and captures the robot's work patterns.

[0225] 2. Data Analysis

[0226] The collected operational and video data is analyzed by the generative AI, which uses this data to analyze the operating status and working patterns of the object and detect any abnormalities.

[0227] For example, if the AI ​​detects an abnormal rise in temperature, it determines whether this is a sign of overheating or abnormal operation.

[0228] 3. Data Retention and Notification

[0229] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[0230] If an anomaly is detected, the server immediately generates an alert and notifies the technician. Alerts are categorized according to the type of anomaly and its urgency.

[0231] Terminal handling

[0232] The terminal is the interface through which the engineer uses the system. The process is explained below in natural language.

[0233] 1. Receiving and displaying data

[0234] The terminal receives analysis results and alert information from the server, allowing engineers to understand the status of the target object in real time.

[0235] The data is displayed visually and visualized on a dashboard as graphs and charts, for example showing temperature or vibration fluctuations over time, allowing engineers to spot anomalies at a glance.

[0236] 2. Managing Notifications

[0237] The device instantly notifies technicians of important alert information in a variety of ways, including push notifications and emails.

[0238] If an abnormality is detected, countermeasures are presented on the terminal, helping the technician to take appropriate measures immediately.

[0239] User Action

[0240] The user, as an engineer, uses the system to manage the operating status and work patterns of the factory robots. The process is explained below in natural language.

[0241] 1. Data Monitoring

[0242] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[0243] For example, if a user is monitoring the temperature status of a factory robot, if an abnormal temperature rise or vibration pattern is detected, the system will notify them and they can quickly take inspection or repair measures.

[0244] 2. Data Management and Feedback

[0245] Users can view long-term data to understand trends in the operating status and work patterns of their equipment, allowing them to provide preventative maintenance.

[0246] If necessary, specific abnormal behaviors or conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[0247] Specific examples

[0248] A specific example is a robot maintenance system in a factory. The server receives temperature and vibration data in real time from sensing devices attached to the robot, and also captures camera footage. The generative AI analyzes this data and monitors the robot's operating status. If an abnormality is detected, an alert is generated and notified to the technician via the device. The technician can immediately analyze the cause of the abnormality and take appropriate action.

[0249] This system makes it possible to efficiently monitor the operating status of factory robots, detect abnormalities early, and respond quickly, thereby improving production efficiency and safety within the factory.

[0250] Prompt Sentence Examples

[0251] "The temperature of a factory robot is rising abnormally. An 85°C warning has been issued. Please provide instructions for generating an immediate alert and notifying a technician."

[0252] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0253] Step 1:

[0254] The server collects operational data such as temperature, vibration, and power consumption in real time from sensing devices attached to factory robots. This input data allows the robot's status to be constantly monitored. The collected data is initially processed on the server and converted into an appropriate format.

[0255] Step 2:

[0256] The server receives video data in real time from cameras installed in the factory. The video data is used to visually capture the robot's movements and work patterns. The received video is divided into frames and saved.

[0257] Step 3:

[0258] The server provides collected operational and video data as input to the generative AI model, which analyzes this data to detect the object's operating status and work patterns. If an abnormality is detected, the AI ​​model will recognize signs such as elevated temperatures or abnormal vibrations.

[0259] Step 4:

[0260] The server stores the analysis results of the generative AI model in a database. The stored data is organized with time information to facilitate future searches and analysis, while maintaining the integrity and security of the data.

[0261] Step 5:

[0262] If the generative AI model detects an anomaly, the server immediately generates an alert, generating an appropriate message depending on the type and urgency of the anomaly, which is then recorded in the alert log.

[0263] Step 6:

[0264] The server notifies the technician of the generated alert. The notification can be sent in the form of a push notification, email, SMS, etc. For example, in the case of a temperature abnormality, a message will be sent saying, "Robot A's temperature has reached 85°C. Immediate action is required."

[0265] Step 7:

[0266] The terminal visually displays the analysis results and alert information received from the server. The results are converted into graphs and charts on the user interface, allowing engineers to check them in real time. For example, fluctuations in temperature or vibration can be seen at a glance.

[0267] Step 8:

[0268] Users use their terminals to monitor and check the analysis results and alert information from the server. When an abnormality is detected, users can provide feedback to the system to take immediate action. Specifically, users can record actions such as "Check that Robot A's temperature is abnormal and check the cooling system."

[0269] Step 9:

[0270] Users can refer to long-term data to understand trends in the operating status and work patterns of their equipment, enabling them to plan preventative maintenance and improvements, thereby improving production efficiency and safety within their factories.

[0271] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0272] System Overview

[0273] This invention is a system for monitoring the health status and behavioral patterns of animals in real time and providing appropriate care. In particular, by combining it with an emotion engine, it is possible to recognize the user's emotional state and dynamically change the notification method and content of alert information based on that. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine, and aims to improve animal welfare and optimize the housing environment.

[0274] Program Description

[0275] Server Processing

[0276] The server is the central hub of the entire system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[0277] 1. Data Collection

[0278] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[0279] The server receives video data from cameras installed in the breeding area.

[0280] Both data are stored in a temporary storage area.

[0281] 2. Data Preprocessing

[0282] The server cleans the collected biometric data and corrects and deletes inaccurate or missing data.

[0283] Noise is also removed from the video data, and frame correction is performed.

[0284] 3. Data Analysis

[0285] The server uses generative AI to analyze the biological and video data, detecting abnormalities in heart rate and body temperature and identifying behavioral patterns.

[0286] For example, if a lion's heart rate suddenly increases, it may be interpreted as a sign of stress or illness.

[0287] 4. Emotional Data Collection and Analysis

[0288] The server uses an emotion engine to collect the user's emotion data and analyze the user's emotional state.

[0289] For example, if the user is feeling stressed, the method of notifying the alert information is changed.

[0290] 5. Data Retention and Notification

[0291] The analysis results are stored in a database, allowing data to be accumulated over a long period of time.

[0292] If an abnormality is detected, the server generates an alert and sends it using a notification method that takes into account the user's emotional state.

[0293] Terminal handling

[0294] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[0295] 1. Receiving and displaying data

[0296] The terminal receives analysis results and alert information from the server in real time.

[0297] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[0298] 2. Displaying the user's emotional state

[0299] The device displays the user's emotional state and visualizes it along with the analysis results. By checking their own emotional state, users can make calm decisions.

[0300] 3. Managing Notifications

[0301] The notification method and content of alert information are dynamically changed depending on the user's emotional state. For example, if the user is feeling stressed, the notification may be changed to email, rather than prompting an immediate response.

[0302] User Action

[0303] Users can use this system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[0304] 1. Data Monitoring

[0305] Users can monitor analysis results and alert information in real time via their terminals.

[0306] The user calmly judges whether there is anything abnormal while checking their own emotional state.

[0307] 2. Implementing emotional coping strategies

[0308] When implementing the displayed countermeasures, the user can take measures that take into account his or her own emotional state.

[0309] For example, it helps a user in a stressed state to take calm action.

[0310] 3. Data Feedback

[0311] Users can provide detailed feedback on abnormal behavior and health conditions to improve the analytical accuracy of the generative AI.

[0312] For example, by recording the progress after treatment and using this information as a reference for the next abnormality detection, response times can be shortened.

[0313] Specific examples

[0314] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. The generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, the emotion engine recognizes the user's emotional state and changes the notification method for alert information. For example, if the user is feeling stressed, an email notification can be sent to avoid immediate response. This system makes it possible to efficiently monitor the health of animals while providing care that takes the user's emotional state into consideration.

[0315] The processing flow will be explained below.

[0316] Server Processing

[0317] Step 1: Collecting data from sensing devices

[0318] server:

[0319] The server receives real-time biological data such as heart rate, body temperature, and location information from sensing devices attached to the animals.

[0320] The collected data is stored in a temporary storage area.

[0321] Step 2: Collecting video data from the camera

[0322] server:

[0323] The server receives video data in real time from cameras installed within the breeding area.

[0324] Video data is also stored in the temporary storage area.

[0325] Step 3: Preprocessing the data

[0326] server:

[0327] The server cleans the biometric data, correcting and deleting inaccurate or missing data.

[0328] Noise removal and frame correction are also performed on the video data.

[0329] Step 4: Analysis of ecological and video data

[0330] server:

[0331] The collected and pre-processed data is analyzed using generative AI to detect abnormalities in heart rate and body temperature.

[0332] Animal behavior patterns are extracted from video data and abnormal behavior is detected.

[0333] Step 5: Emotional Data Collection and Analysis

[0334] server:

[0335] The emotion engine collects emotion data from the user's device, including facial expression recognition and voice analysis.

[0336] The server analyzes the collected emotional data and determines the user's stress level and level of urgency.

[0337] Step 6: Data storage and alert generation

[0338] server:

[0339] The analysis results are stored in a database, and data is accumulated over a long period of time.

[0340] If an abnormality is detected, an alert is generated, and the alert information includes the type of abnormality and its urgency.

[0341] Step 7: Sending notifications

[0342] server:

[0343] A notification method for alert information is dynamically selected based on the emotional state of a user.

[0344] For example, if the user is feeling stressed, the notification method can be changed to email, and immediate action will not be urged.

[0345] Terminal handling

[0346] Step 1: Receiving Data

[0347] Device:

[0348] The terminal receives analysis results and alert information from the server in real time.

[0349] Step 2: Visualize the data

[0350] Device:

[0351] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[0352] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[0353] Step 3: Obtaining and displaying emotion data

[0354] Device:

[0355] The device analyzes and displays the user's emotional state in real time, using a camera and microphone to collect emotional data.

[0356] The user's current emotional state is displayed and visualized along with the analysis results.

[0357] Step 4: Manage notifications

[0358] Device:

[0359] The notification method and content of alert information are dynamically changed according to the user's emotional state.

[0360] For example, if the user is feeling stressed, the system will notify them in a quieter way, avoiding the audible alarm.

[0361] User Action

[0362] Step 1: Data monitoring

[0363] User:

[0364] Users can monitor analysis results and alert information in real time via their terminals.

[0365] Check your emotional state in conjunction with the alert information and calmly assess any abnormalities.

[0366] Step 2: Implementing countermeasures

[0367] User:

[0368] When implementing the displayed countermeasures, you can take into account your own emotional state.

[0369] For example, it helps a user in a stressed state to take calm action.

[0370] Step 3: Data feedback

[0371] User:

[0372] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[0373] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities next time.

[0374] Based on this detailed processing flow, a series of processes from collecting animal biological data to analysis, notification, and response are reliably carried out while taking into account the emotional state of the user.

[0375] Example 2

[0376] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0377] Conventional animal health and behavioral pattern monitoring systems only detect abnormalities in animals and do not provide a notification method that takes into account the user's emotional state. As a result, if the user is feeling stressed, the notification may not be received properly, making it difficult to respond quickly. Furthermore, there is no way for the user to understand their own emotional state and deal with the situation calmly.

[0378] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0379] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI to detect changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, means for analyzing the user's emotional state, and means for dynamically changing the alert notification method based on the user's emotional state. This enables efficient monitoring of the animal's health condition while enabling prompt and appropriate responses by using a notification method that takes the user's emotional state into consideration.

[0380] "Sensing device" refers to a device worn by an animal to collect biological data in real time.

[0381] "Biological Data" refers to information including physiological and behavioral data of an animal, such as the animal's heart rate, body temperature, and location.

[0382] "Video data" refers to video information of animal behavior and environment collected using a camera or other image capture device.

[0383] "Generative AI" refers to an artificial intelligence model for analyzing ecological and video data. This model has the ability to automatically analyze animal health and behavioral patterns.

[0384] A "database" refers to a data management system for storing analysis results and collected data, which allows for long-term data accumulation.

[0385] An "alert" is a notification that is generated when an abnormality is detected. This notification notifies the user that an abnormality has occurred and prompts them to take appropriate action.

[0386] "User emotional state" refers to data that indicates the psychological state and stress level of the user using the system.

[0387] "Emotion engine" refers to a system component that has the function of collecting and analyzing the user's emotional state.

[0388] A "dashboard" refers to a user interface that allows users to visually check analysis results and their own emotional state.

[0389] "Dynamic change of notification method" refers to changing the notification method and content of alert information in real time depending on the user's emotional state. Examples include email notifications and on-screen pop-up notifications.

[0390] Specific embodiments of the present invention are described below.

[0391] This system monitors the health and behavioral patterns of animals in real time and provides appropriate care. In particular, by combining it with an emotion engine, it can recognize the user's emotional state and dynamically change the appropriate notification method. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine.

[0392] The server serves as the central hub of the entire system and performs the following processes: First, the server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals. These sensing devices can be ordinary heart rate monitors or temperature sensors. The server also streams video data from cameras installed in the animal enclosures and stores it in a temporary storage area. This video data is used to analyze the behavioral patterns of the animals.

[0393] The server then cleans the collected ecology data, correcting and deleting inaccurate or missing data. It also removes noise from the video data and performs frame correction to improve the accuracy of the analysis. Generative AI is then used to analyze the ecology data and video data. This generative AI detects abnormal values ​​in the animal's heart rate and body temperature and identifies behavioral patterns.

[0394] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. For example, if a lion's heart rate suddenly rises, the server will interpret this as a sign of stress or illness.

[0395] The server also collects the user's emotional state through the user interface. The emotion engine evaluates the user's stress level based on the user's operation speed and response time. Based on this information, the server analyzes the user's emotional state and dynamically changes the notification method for alert information. For example, if the user is feeling stressed, the server can change the emergency alert notification to an email notification, avoiding the need for an immediate response.

[0396] The analysis results are stored in a database, allowing data to be accumulated over a long period of time. This allows for long-term monitoring of the animal's health. If an abnormality is detected, the server generates an alert and sends the alert information using a notification method that takes the user's emotional state into consideration.

[0397] The terminal is the interface through which users use the system, receiving analysis results and alert information from the server in real time. To visually display the received data, the terminal generates graphs and charts and provides information to the user. Furthermore, the terminal displays the user's emotional state, which is visualized together with the analysis results, helping the user to calmly determine whether an abnormality has occurred. The terminal also appropriately manages the notification method and content of alert information according to the user's emotional state.

[0398] Finally, users can use the system to manage their animals' health and behavioral patterns. They can monitor analysis results and alert information in real time, and calmly assess abnormalities while checking their own emotional state. Furthermore, users can provide detailed feedback on abnormal behavior and health conditions to improve the analysis accuracy of the generated AI.

[0399] Examples of prompts include:

[0400] "Lion's heart rate is significantly higher than normal. If this abnormality persists, immediate medical attention is required. Due to the user's high stress levels, a notification has been sent via email."

[0401] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0402] Step 1:

[0403] Data collection

[0404] Server operation: The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals, as well as video data from cameras installed in the animal enclosures.

[0405] Input: Biological data from sensing devices and video data from cameras.

[0406] Output: Biological data and video data stored in temporary storage area.

[0407] Specific operation: The server collects data from the sensing device every 5 seconds and stores it in a temporary storage area. It also streams camera footage and stores it in the temporary storage area.

[0408] Step 2:

[0409] Data Preprocessing

[0410] Server operation: The server cleans the collected biological data, corrects and deletes inaccurate and missing data, and removes noise from video data and performs frame correction.

[0411] Input: Biological data and video data in temporary storage area.

[0412] Output: Preprocessed and clean data.

[0413] Specific operation: The server executes a data cleaning algorithm to remove inaccurate points in the biological data and applies a noise reduction filter to the video data.

[0414] Step 3:

[0415] Data analysis

[0416] Server operation: The server uses generative AI to analyze the ecology and video data, specifically detecting abnormalities in heart rate and body temperature and identifying animal behavior patterns.

[0417] Input: Preprocessed ecological data and video data.

[0418] Output: Anomaly detection report and specific information on behavioral patterns as analysis results.

[0419] Specific operation: The generative AI model is executed to generate alert data when the heart rate exceeds a certain range or when abnormal movement is detected from the video.

[0420] Step 4:

[0421] Emotion data collection and analysis

[0422] Server operation: The server uses an emotion engine to collect and analyze user emotion data. It monitors the user's operation speed and response time to evaluate their stress level.

[0423] Input: User operation data and response data.

[0424] Output: Parsed user emotional state data.

[0425] Specific operation: The emotion engine analyzes the log data of the user interface and quantifies the user's stress level.

[0426] Step 5:

[0427] Data Retention and Notification

[0428] Server operation: The server stores the analysis results in a database and generates an alert if an anomaly is detected. The alert dynamically changes the notification method taking into account the user's emotional state.

[0429] Input: Anomaly detection report, emotional state data.

[0430] Output: Data stored in the database and alert information sent.

[0431] Specific operation: The server saves the anomaly detection report in a database, and based on the analysis results of the emotion engine, selects an appropriate notification method, such as email or a pop-up notification on the screen, and sends an alert to the user.

[0432] Step 6:

[0433] User Interface Operations

[0434] Device operation: The device receives analysis results and alert information from the server in real time and visually displays them, as well as the user's emotional state.

[0435] Input: Analysis results and alert information from the server.

[0436] Output: Visualized analysis results, alert information, and user emotional state.

[0437] Specific operation: The device converts the received data into graphs and charts and displays them, showing the user's emotional state in real time.

[0438] Step 7:

[0439] Data Feedback

[0440] User Action: The user uses the system to monitor the health and behavioral patterns of the animals and provides detailed feedback on abnormal behavior and health status.

[0441] Input: Analysis results and alert information.

[0442] Output: Feedback data from users.

[0443] Specific operation: The user monitors the system and, if an abnormality is detected, takes countermeasures and inputs the results into the system as feedback.

[0444] (Application example 2)

[0445] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0446] Currently, there are systems that monitor the health and behavioral patterns of animals in real time, but they do not take into account the user's emotional state. This results in inappropriate notifications being sent when the user feels stressed, resulting in an inability to respond appropriately. Furthermore, there is a lack of technology that can effectively analyze biological data from sensors and camera footage to quickly detect abnormalities. Therefore, there is a need for flexible notifications that take into account the user's emotional state while effectively managing animal health.

[0447] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting animal ecology data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecology data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, and means for dynamically changing the notification method based on the user's emotional state using an emotion engine that collects and analyzes the user's emotional data. This enables effective monitoring of the animal's health condition and flexible notification that takes the user's emotional state into consideration.

[0448] A "sensing device" is a device used to collect biological and operational data of animals and machines in real time.

[0449] "Video data" means visually recorded data of animal or machine behavior obtained by a camera or other visual sensor.

[0450] "Generative AI" is a technology that uses artificial intelligence techniques to analyze large amounts of data and detect abnormalities in health conditions and changes in behavioral patterns.

[0451] The "database" is a system for storing and managing analysis results, ecological data, and video data over the long term.

[0452] An "alert" is a warning message or signal that notifies the user when an abnormality is detected.

[0453] An "emotion engine" is a technology for analyzing a user's emotional state and assessing the user's emotions based on collected data.

[0454] A "prompt sentence" is a sentence used to explain to the user specific analysis results or abnormal behavior generated by the generative AI.

[0455] The "dashboard" is an interface that allows users to visually check, monitor, and manage analysis results and emotional states.

[0456] "Notification method" refers to the means by which alerts and information are communicated to users, such as email or instant messaging.

[0457] This invention provides a system that monitors the health status and behavior patterns of animals or machines in real time and provides effective countermeasures taking into account the emotional state of the user. The system includes a sensing device, a camera, a generative AI, an emotion engine, a database, a dashboard, and a notification means.

[0458] System Configuration

[0459] 1. Sensing Devices

[0460] This device is attached to animals or machines and collects biological and behavioral data such as heart rate, body temperature, and vibration in real time. The collected data is sent to a server.

[0461] 2. Camera

[0462] This device captures the target's movement patterns as video data, which is then sent to a server for analysis.

[0463] 3. Server

[0464] It is the central part of the entire system and performs the following processes:

[0465] Data collection from sensing devices and cameras.

[0466] Pre-processing of collected data (noise removal, data correction).

[0467] Data analysis using generative AI.

[0468] Generate alerts when anomalies are detected.

[0469] Analyzing user emotional states using an emotion engine.

[0470] Analysis results and alert information are stored in a database and sent to users.

[0471] Program processing explanation

[0472] 1. Data Collection

[0473] The server receives real-time data such as heart rate, body temperature, and vibration from sensing devices attached to animals and machines. It also receives video data from cameras installed in the breeding area and factory. This data is stored in a temporary storage area.

[0474] 2. Data Preprocessing

[0475] The server cleanses the collected data, correcting and deleting inaccurate or missing data, and also removes noise from video data and performs frame correction.

[0476] 3. Data Analysis

[0477] The server uses generative AI to analyze the biological data and video data. For example, if the temperature sensor data suddenly rises, it may indicate that the machine is overheating. This allows the server to detect changes in the health and behavior patterns of animals and machines.

[0478] 4. Emotional Data Collection and Analysis

[0479] The emotion engine collects user emotional data and analyzes their emotional state, allowing the system to take appropriate measures such as changing notification methods if the user is feeling stressed.

[0480] 5. Data Retention and Notification

[0481] The server stores the analysis results in a database and generates an alert if an abnormality is detected. The emotion engine takes the user's emotional state into account and dynamically changes the notification method and content. For example, if the user is feeling stressed, they can select to be notified by email.

[0482] Specific examples

[0483] In a factory application, the operating status of a robot is monitored. Data from vibration sensors and video data from cameras are sent to a server from sensing devices. A generative AI model analyzes this data and detects anomalies. An emotion engine evaluates the user's emotional state and selects an email notification if, for example, the user is experiencing high stress.

[0484] An example prompt is:

[0485] Sensor data: Temperature is over 80 degrees.

[0486] Camera footage analysis: Robot movement is unstable.

[0487] Emotion Engine Input: The user is concerned about the high temperature data from the sensor and the results of the video analysis.

[0488] This system allows for efficient health management of animals or machines, while also providing flexible responses that take into account the emotional state of the user.

[0489] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0490] Step 1: Data collection

[0491] The server receives biological data such as heart rate, body temperature, and vibration in real time from sensing devices attached to animals and machines. It also receives video data from cameras of the breeding areas and factory interiors. The input is numerical data of heart rate and body temperature and video data, and the output is raw data stored in a temporary storage area. Specifically, it retrieves data from sensors and cameras using HTTP requests.

[0492] Step 2: Preprocessing the data

[0493] The server cleans the collected biological data and video data, correcting and deleting inaccurate and missing data. It also performs noise removal and frame correction on video data. The input is raw data stored in a temporary storage area, and the output is cleaned, consistent data. Specific operations include applying data cleaning algorithms and noise removal filters.

[0494] Step 3: Data analysis

[0495] The server uses the generative AI model to analyze the cleaned biological data and video data. This detects abnormal values ​​in heart rate and body temperature and identifies changes in behavior patterns. The input is the pre-processed data, and the output is the analysis results. Specifically, the data is input into the generative AI model and the analysis results are obtained.

[0496] Step 4: Collect and analyze emotion data

[0497] The server uses an emotion engine to collect and analyze the user's emotional data. It evaluates the user's emotional state based on the text data and physical condition data entered into the system. The input is the user's text data and physical condition data, and the output is the analysis result of the user's emotional state. Specifically, the text and physical condition data are entered into the emotion engine, and an emotion evaluation result is obtained.

[0498] Step 5: Alerting and Notification

[0499] The server generates an alert if an anomaly is detected based on the results of data analysis and sentiment analysis. Furthermore, it uses an emotion engine to select a notification method that takes into account the user's emotional state, and notifies the user of the alert information. The input is the results of data analysis and sentiment analysis, and the output is the alert information sent to the user. Specifically, the server uses an anomaly detection algorithm to generate an alert and sends the notification via the selected method (email or instant message).

[0500] Step 6: Data storage and feedback

[0501] The server stores the analysis results and alert information in a database. Users can check the displayed information and provide feedback to help improve the accuracy of the generative AI model and emotion engine. The input is the analysis results and user feedback, and the output is an updated database. Specific operations include writing the analysis results and integrating the feedback data.

[0502] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0503] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0504] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0505] [Second embodiment]

[0506] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0507] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0508] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0509] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0510] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0511] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0512] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0513] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0514] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0515] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0516] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0517] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0518] System Overview

[0519] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[0520] Program Description

[0521] Server Processing

[0522] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[0523] 1. Data Collection

[0524] The server receives real-time biological data such as heart rate, body temperature, and location information from the sensing devices attached to the animals, allowing the server to instantly grasp the animal's condition.

[0525] The server receives video data from cameras installed in the breeding area and captures the behavior of the animals.

[0526] 2. Data Analysis

[0527] The collected ecological and video data is analyzed by generative AI, which uses this data to analyze the animals' health and behavioral patterns and detect any abnormalities.

[0528] For example, if the AI ​​detects an abnormally elevated heart rate, it will determine whether this is a sign of stress or illness.

[0529] 3. Data Retention and Notification

[0530] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[0531] If an abnormality is detected, the server immediately generates an alert and notifies the user. Alerts are classified according to the type and urgency of the abnormality.

[0532] Terminal handling

[0533] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[0534] 1. Receiving and displaying data

[0535] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[0536] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[0537] 2. Managing Notifications

[0538] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[0539] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[0540] User Action

[0541] Users use the system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[0542] 1. Data Monitoring

[0543] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[0544] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[0545] 2. Data Management and Feedback

[0546] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[0547] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[0548] Specific examples

[0549] A specific example is a health management system for lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action.

[0550] This system allows for efficient monitoring of animal health, early detection of abnormalities, and rapid response, thereby improving animal welfare and optimizing the breeding environment.

[0551] The processing flow will be explained below.

[0552] Server Processing

[0553] Step 1: Data collection

[0554] server:

[0555] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[0556] The server receives video data from cameras installed in the breeding area.

[0557] The collected data is stored in a temporary storage area.

[0558] Step 2: Preprocessing the data

[0559] server:

[0560] The server cleans the collected biological data, correcting and deleting inaccurate or missing data.

[0561] Noise removal and frame correction are also performed on the video data.

[0562] Step 3: Data analysis

[0563] server:

[0564] The server uses generative AI to analyze biological data in real time, detecting abnormalities in heart rate and body temperature.

[0565] The server uses video analysis tools to analyze the animal's behavioral patterns and detect abnormal behavior.

[0566] Step 4: Save Data

[0567] server:

[0568] The analysis results are stored in a database, which allows data to be accumulated over a long period of time.

[0569] The data will be organized chronologically to allow for quick searching and reference.

[0570] Step 5: Alerting and Notification

[0571] server:

[0572] If an anomaly is detected, the server immediately generates an alert, which includes the type of anomaly and its urgency.

[0573] Sends generated alerts to the device.

[0574] Terminal handling

[0575] Step 1: Receiving Data

[0576] Device:

[0577] The terminal receives analysis results and alert information from the server in real time.

[0578] Step 2: Visualize the data

[0579] Device:

[0580] Generate graphs and charts to visually display the received data, such as changes in heart rate or body temperature.

[0581] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[0582] Step 3: Sending notifications

[0583] Device:

[0584] Alert information is notified to the user via push notification, email, etc.

[0585] User Action

[0586] Step 1: Data monitoring

[0587] User:

[0588] Users can monitor analysis results and alert information in real time via their terminals.

[0589] If an important alert occurs, immediately check the notification and take the necessary action.

[0590] Step 2: Implementing countermeasures

[0591] User:

[0592] Based on the countermeasures presented by the server, appropriate care is provided for the animal's health condition and behavioral patterns.

[0593] For example, if an abnormality is detected, a veterinarian will be contacted and a detailed health check will be requested.

[0594] Step 3: Data feedback

[0595] User:

[0596] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[0597] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities in the future.

[0598] In this way, the entire process from collecting animal ecological data to analysis, notification, and response is carried out reliably.

[0599] Example 1

[0600] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] Modern animal husbandry environments require effective systems for real-time monitoring of animal health and behavioral patterns and early detection of abnormalities. Conventional methods have fragmented processes for data collection, analysis, and notification, making it difficult to respond immediately. They also lack an interface that allows users to intuitively grasp the health status of animals. To solve these problems and improve animal welfare, an integrated system must be built.

[0602] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0603] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for sending the generated alert to a terminal and the terminal notifying the user of it in real time, means for visually displaying the analysis results through a user interface and visualizing them as graphs or charts, and means for the user to refer to past data and analyze trends. This makes it possible to efficiently monitor the animal's health condition and behavioral patterns, detect abnormalities early, and take appropriate measures promptly.

[0604] A "sensing device" is a device that is attached to an animal and collects biological data such as heart rate, body temperature, and location information in real time.

[0605] "Video data" refers to digital data that records the movements and behavior of animals, captured using a video device such as a camera.

[0606] "Generative AI" is an artificial intelligence technology that analyzes ecological and video data to detect changes in animals' health and behavioral patterns.

[0607] A "database" is an information storage system for saving analysis results, and provides long-term data accumulation and backup functions.

[0608] An "alert" is a warning message that is generated when an abnormality is detected, and is a means of notifying the user.

[0609] A "terminal" is an interface device that allows users to use the system and receives and displays alerts and analysis results from the server.

[0610] A "user interface" is a collection of screens and functions that contain visual and operational elements that allow a user to interact with a system.

[0611] A "graph" is a diagram for visually displaying data, visualizing fluctuations along a time axis.

[0612] A "chart" is a graphic representation of numerical data visually, and includes formats such as bar graphs and line graphs.

[0613] A "trend" refers to a tendency or tendency observed from past data.

[0614] System Overview

[0615] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[0616] Program Description

[0617] Server Processing

[0618] The server is the center of this system, and performs data collection, analysis, storage, and notification. Specifically, it performs the following processes:

[0619] 1. Data Collection

[0620] The server receives real-time biological data, such as heart rate, body temperature, and location information, from sensing devices attached to the animals. For example, it receives data that a lion's heart rate has increased from 60 bpm to 80 bpm.

[0621] The server receives video data from cameras installed in the enclosure, capturing footage of the lions' movements and postures every 10 seconds.

[0622] 2. Data Transmission

[0623] The server simultaneously receives the collected biological data and video data, so it needs to maintain stable data communication over the network. For example, the server synchronizes data with the device using Wi-Fi.

[0624] 3. Data Analysis

[0625] The generative AI model on the server analyzes the received biological data and video data in real time. Specifically, the AI ​​detects abnormal heart rate patterns and behavioral abnormalities based on the video data. For example, it detects stress by linking a sudden increase in heart rate with vigorous movements in the video data.

[0626] 4. Data storage and alert generation

[0627] The server stores the results of the analysis by the generative AI in a database. For example, the analysis results are stored in a database as a lion's health log, including heart rate, body temperature, and behavioral patterns.

[0628] The server generates an alert if it detects an abnormality. For example, if the heart rate is abnormally high, the server adds "Abnormal Heart Rate: Urgent" to the alert list.

[0629] 5. User Notices

[0630] The server sends the generated alerts to the device. For example, when a new alert is generated, the server notifies the user by push notification or email.

[0631] The device notifies the user of the received alert in real time. For example, the device may use the smartphone's push notification function to send the user a message saying, "The lion's heart rate is abnormal. Please check immediately."

[0632] Terminal handling

[0633] The terminal is an interface that allows users to use the system. Specifically, it includes the following processes:

[0634] 1. Receiving and displaying data

[0635] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[0636] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[0637] 2. Managing Notifications

[0638] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[0639] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[0640] User Action

[0641] Users use the system to manage the health and behavioral patterns of their animals, including the following processes:

[0642] 1. Data Monitoring

[0643] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[0644] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[0645] 2. Data Management and Feedback

[0646] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[0647] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[0648] Specific examples

[0649] A specific example is a health management system for lions at a zoo. The server receives heart rate and body temperature data in real time from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action. This system makes it possible to efficiently monitor the health of animals, detect abnormalities early, and respond quickly. This improves animal welfare and optimizes the captive environment.

[0650] Prompt Sentence Examples

[0651] "Analyze the heart rate and temperature data received from the sensing device attached to the lion, as well as the camera footage, to assess the lion's health and behavioral patterns. If you detect any abnormalities, state the reason and recommend a course of action."

[0652] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0653] Step 1:

[0654] Data collection

[0655] The server receives biological data such as heart rate, body temperature, and location information in real time from sensing devices attached to the animals. The input is data from various sensors, and the output is raw data stored in the server. Specifically, the server receives and temporarily stores data on heart rate increases from 60 bpm to 80 bpm and body temperature data every unit of time.

[0656] Step 2:

[0657] Video data collection

[0658] The server receives video data from cameras installed in the enclosure. The input is the video signal from the camera, and the output is the video data stored in the server. For example, the server captures and stores video of lions' movements every 10 seconds.

[0659] Step 3:

[0660] Data transmission

[0661] The server communicates via a network to stably receive the collected biological data and video data. The input is data from the sensing device and camera, and the output is the integrated data sent to the server. Specifically, the server synchronizes the real-time data using Wi-Fi.

[0662] Step 4:

[0663] Data analysis

[0664] The generative AI model on the server analyzes the received ecological data and video data in real time. The input is the ecological data and video data stored on the server, and the output is the analyzed health status and behavioral pattern changes. Specifically, the AI ​​analyzes sudden increases in heart rate and violent animal movements based on the video to detect stress or health abnormalities.

[0665] Step 5:

[0666] Data storage

[0667] The server stores the results of the analysis by the generative AI in a database. The input is the analyzed health data and behavioral pattern data, and the output is the historical data stored in the database. Specifically, the server adds logs of the lion's heart rate, body temperature, and behavioral patterns to the database.

[0668] Step 6:

[0669] Alert Generation

[0670] The server generates an alert if it detects an abnormality. The input is the abnormal data from the generation AI, and the output is a warning message added to the alert list. For example, if the heart rate is abnormally high, the server adds "Heart rate abnormal: emergency" to the alert list.

[0671] Step 7:

[0672] Alert Notification

[0673] The server sends the generated alert to the device. The input is the alert message, and the output is the alert notification sent to the device. Specifically, the server sends an emergency message to the user via push notification or email.

[0674] Step 8:

[0675] Real-time notifications

[0676] The terminal notifies the user of the received alert in real time. The input is the alert notification sent from the server, and the output is a warning message displayed on the user's terminal. For example, the terminal may display a message such as "The lion's heart rate is abnormal. Please check immediately."

[0677] Step 9:

[0678] Data Monitoring

[0679] The user uses a terminal to monitor the analysis results and alert information from the server in real time. The input is the analysis results displayed on the terminal's dashboard, and the output is the user's judgment and response. Specifically, the user looks at the graphs and charts on the terminal to identify abnormalities.

[0680] Step 10:

[0681] Data Management and Feedback

[0682] Users can view long-term data to understand trends in animal health and behavior. The input is historical database information, and the output is analysis results and feedback. Specifically, users can view a year's worth of health data in graphs and provide feedback to the system to improve the accuracy of the generative AI.

[0683] (Application example 1)

[0684] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] In recent years, as the complexity and operating hours of robots used in factories have increased, it has become increasingly important to monitor the status of robots in real time and detect abnormalities early. However, current systems lack a mechanism for immediately detecting and efficiently notifying robot abnormalities, which is a factor that affects production efficiency and safety. A solution to this issue is needed.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0687] In this invention, the server includes means for collecting operational data of the object from the sensing device in real time, means for receiving video data and analyzing the work pattern of the object, means for analyzing the operational data and video data using a generation AI and detecting changes in the operating state and work pattern of the object, means for saving the analysis results in a database and generating an alert when an abnormality is detected, and means for notifying engineers of the analysis results and alert information. This makes it possible to monitor robot abnormalities in real time and to immediately notify engineers when an abnormality is detected.

[0688] A "sensing device" is a device that collects operational data and environmental data (temperature, vibration, power consumption, etc.) of an object in real time.

[0689] "Operation data" refers to data on the status of an object while it is in operation (such as temperature, vibration, and power consumption).

[0690] "Video data" refers to video information of an object being worked on, captured using photographic equipment such as a camera.

[0691] "Work pattern" refers to the pattern of actions and behaviors of an object when performing a specific task or operation.

[0692] "Generative AI" is artificial intelligence that analyzes changes in the operating status and work patterns of objects based on collected operational data and video data.

[0693] "Analysis results" refer to the information and insights obtained when operational data and video data are analyzed by the generating AI.

[0694] A "database" is a storage medium or system for long-term storage and management of analysis results and ecological data.

[0695] An "alert" is a warning notification that is generated when an abnormality is detected and is immediately reported to a technician.

[0696] An "engineer" is someone who receives analysis results and alert information and takes action.

[0697] A "dashboard" is a user interface that allows engineers to visually check analysis results and alert information.

[0698] System Overview

[0699] This invention is a system that enables preventive maintenance by monitoring the status and operation patterns of factory robots in real time and immediately notifying engineers if an abnormality is detected. The system mainly consists of a sensing device, a video analysis tool, a generative AI, and a user interface with a dashboard.

[0700] Program Description

[0701] Server Processing

[0702] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[0703] 1. Data Collection

[0704] The server receives real-time operational data such as temperature, vibration, and power consumption from sensing devices attached to factory robots, allowing the status of the target object to be grasped immediately.

[0705] The server receives video data from cameras installed in the factory and captures the robot's work patterns.

[0706] 2. Data Analysis

[0707] The collected operational and video data is analyzed by the generative AI, which uses this data to analyze the operating status and working patterns of the object and detect any abnormalities.

[0708] For example, if the AI ​​detects an abnormal rise in temperature, it determines whether this is a sign of overheating or abnormal operation.

[0709] 3. Data Retention and Notification

[0710] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[0711] If an anomaly is detected, the server immediately generates an alert and notifies the technician. Alerts are categorized according to the type of anomaly and its urgency.

[0712] Terminal handling

[0713] The terminal is the interface through which the engineer uses the system. The process is explained below in natural language.

[0714] 1. Receiving and displaying data

[0715] The terminal receives analysis results and alert information from the server, allowing engineers to understand the status of the target object in real time.

[0716] The data is displayed visually and visualized on a dashboard as graphs and charts, for example showing temperature or vibration fluctuations over time, allowing engineers to spot anomalies at a glance.

[0717] 2. Managing Notifications

[0718] The device instantly notifies technicians of important alert information in a variety of ways, including push notifications and emails.

[0719] If an abnormality is detected, countermeasures are presented on the terminal, helping the technician to take appropriate measures immediately.

[0720] User Action

[0721] The user, as an engineer, uses the system to manage the operating status and work patterns of the factory robots. The process is explained below in natural language.

[0722] 1. Data Monitoring

[0723] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[0724] For example, if a user is monitoring the temperature status of a factory robot, if an abnormal temperature rise or vibration pattern is detected, the system will notify them and they can quickly take inspection or repair measures.

[0725] 2. Data Management and Feedback

[0726] Users can view long-term data to understand trends in the operating status and work patterns of their equipment, allowing them to provide preventative maintenance.

[0727] If necessary, specific abnormal behaviors or conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[0728] Specific examples

[0729] A specific example is a robot maintenance system in a factory. The server receives temperature and vibration data in real time from sensing devices attached to the robot, and also captures camera footage. The generative AI analyzes this data and monitors the robot's operating status. If an abnormality is detected, an alert is generated and notified to the technician via the device. The technician can immediately analyze the cause of the abnormality and take appropriate action.

[0730] This system makes it possible to efficiently monitor the operating status of factory robots, detect abnormalities early, and respond quickly, thereby improving production efficiency and safety within the factory.

[0731] Prompt Sentence Examples

[0732] "The temperature of a factory robot is rising abnormally. An 85°C warning has been issued. Please provide instructions for generating an immediate alert and notifying a technician."

[0733] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0734] Step 1:

[0735] The server collects operational data such as temperature, vibration, and power consumption in real time from sensing devices attached to factory robots. This input data allows the robot's status to be constantly monitored. The collected data is initially processed on the server and converted into an appropriate format.

[0736] Step 2:

[0737] The server receives video data in real time from cameras installed in the factory. The video data is used to visually capture the robot's movements and work patterns. The received video is divided into frames and saved.

[0738] Step 3:

[0739] The server provides collected operational and video data as input to the generative AI model, which analyzes this data to detect the object's operating status and work patterns. If an abnormality is detected, the AI ​​model will recognize signs such as elevated temperatures or abnormal vibrations.

[0740] Step 4:

[0741] The server stores the analysis results of the generative AI model in a database. The stored data is organized with time information to facilitate future searches and analysis, while maintaining the integrity and security of the data.

[0742] Step 5:

[0743] If the generative AI model detects an anomaly, the server immediately generates an alert, generating an appropriate message depending on the type and urgency of the anomaly, which is then recorded in the alert log.

[0744] Step 6:

[0745] The server notifies the technician of the generated alert. The notification can be sent in the form of a push notification, email, SMS, etc. For example, in the case of a temperature abnormality, a message will be sent saying, "Robot A's temperature has reached 85°C. Immediate action is required."

[0746] Step 7:

[0747] The terminal visually displays the analysis results and alert information received from the server. The results are converted into graphs and charts on the user interface, allowing engineers to check them in real time. For example, fluctuations in temperature or vibration can be seen at a glance.

[0748] Step 8:

[0749] Users use their terminals to monitor and check the analysis results and alert information from the server. When an abnormality is detected, users can provide feedback to the system to take immediate action. Specifically, users can record actions such as "Check that Robot A's temperature is abnormal and check the cooling system."

[0750] Step 9:

[0751] Users can refer to long-term data to understand trends in the operating status and work patterns of their equipment, enabling them to plan preventative maintenance and improvements, thereby improving production efficiency and safety within their factories.

[0752] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0753] System Overview

[0754] This invention is a system for monitoring the health status and behavioral patterns of animals in real time and providing appropriate care. In particular, by combining it with an emotion engine, it is possible to recognize the user's emotional state and dynamically change the notification method and content of alert information based on that. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine, and aims to improve animal welfare and optimize the housing environment.

[0755] Program Description

[0756] Server Processing

[0757] The server is the central hub of the entire system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[0758] 1. Data Collection

[0759] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[0760] The server receives video data from cameras installed in the breeding area.

[0761] Both data are stored in a temporary storage area.

[0762] 2. Data Preprocessing

[0763] The server cleans the collected biometric data and corrects and deletes inaccurate or missing data.

[0764] Noise is also removed from the video data, and frame correction is performed.

[0765] 3. Data Analysis

[0766] The server uses generative AI to analyze the biological and video data, detecting abnormalities in heart rate and body temperature and identifying behavioral patterns.

[0767] For example, if a lion's heart rate suddenly increases, it may be interpreted as a sign of stress or illness.

[0768] 4. Emotional Data Collection and Analysis

[0769] The server uses an emotion engine to collect the user's emotion data and analyze the user's emotional state.

[0770] For example, if the user is feeling stressed, the method of notifying the alert information is changed.

[0771] 5. Data Retention and Notification

[0772] The analysis results are stored in a database, allowing data to be accumulated over a long period of time.

[0773] If an abnormality is detected, the server generates an alert and sends it using a notification method that takes into account the user's emotional state.

[0774] Terminal handling

[0775] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[0776] 1. Receiving and displaying data

[0777] The terminal receives analysis results and alert information from the server in real time.

[0778] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[0779] 2. Displaying the user's emotional state

[0780] The device displays the user's emotional state and visualizes it along with the analysis results. By checking their own emotional state, users can make calm decisions.

[0781] 3. Managing Notifications

[0782] The notification method and content of alert information are dynamically changed depending on the user's emotional state. For example, if the user is feeling stressed, the notification may be changed to email, rather than prompting an immediate response.

[0783] User Action

[0784] Users can use this system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[0785] 1. Data Monitoring

[0786] Users can monitor analysis results and alert information in real time via their terminals.

[0787] The user calmly judges whether there is anything abnormal while checking their own emotional state.

[0788] 2. Implementing emotional coping strategies

[0789] When implementing the displayed countermeasures, the user can take measures that take into account his or her own emotional state.

[0790] For example, it helps a user in a stressed state to take calm action.

[0791] 3. Data Feedback

[0792] Users can provide detailed feedback on abnormal behavior and health conditions to improve the analytical accuracy of the generative AI.

[0793] For example, by recording the progress after treatment and using this information as a reference for the next abnormality detection, response times can be shortened.

[0794] Specific examples

[0795] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. The generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, the emotion engine recognizes the user's emotional state and changes the notification method for alert information. For example, if the user is feeling stressed, an email notification can be sent to avoid immediate response. This system makes it possible to efficiently monitor the health of animals while providing care that takes the user's emotional state into consideration.

[0796] The processing flow will be explained below.

[0797] Server Processing

[0798] Step 1: Collecting data from sensing devices

[0799] server:

[0800] The server receives real-time biological data such as heart rate, body temperature, and location information from sensing devices attached to the animals.

[0801] The collected data is stored in a temporary storage area.

[0802] Step 2: Collecting video data from the camera

[0803] server:

[0804] The server receives video data in real time from cameras installed within the breeding area.

[0805] Video data is also stored in the temporary storage area.

[0806] Step 3: Preprocessing the data

[0807] server:

[0808] The server cleans the biometric data, correcting and deleting inaccurate or missing data.

[0809] Noise removal and frame correction are also performed on the video data.

[0810] Step 4: Analysis of ecological and video data

[0811] server:

[0812] The collected and pre-processed data is analyzed using generative AI to detect abnormalities in heart rate and body temperature.

[0813] Animal behavior patterns are extracted from video data and abnormal behavior is detected.

[0814] Step 5: Emotional Data Collection and Analysis

[0815] server:

[0816] The emotion engine collects emotion data from the user's device, including facial expression recognition and voice analysis.

[0817] The server analyzes the collected emotional data and determines the user's stress level and level of urgency.

[0818] Step 6: Data storage and alert generation

[0819] server:

[0820] The analysis results are stored in a database, and data is accumulated over a long period of time.

[0821] If an abnormality is detected, an alert is generated, and the alert information includes the type of abnormality and its urgency.

[0822] Step 7: Sending notifications

[0823] server:

[0824] A notification method for alert information is dynamically selected based on the emotional state of a user.

[0825] For example, if the user is feeling stressed, the notification method can be changed to email, and immediate action will not be urged.

[0826] Terminal handling

[0827] Step 1: Receiving Data

[0828] Device:

[0829] The terminal receives analysis results and alert information from the server in real time.

[0830] Step 2: Visualize the data

[0831] Device:

[0832] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[0833] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[0834] Step 3: Obtaining and displaying emotion data

[0835] Device:

[0836] The device analyzes and displays the user's emotional state in real time, using a camera and microphone to collect emotional data.

[0837] The user's current emotional state is displayed and visualized along with the analysis results.

[0838] Step 4: Manage notifications

[0839] Device:

[0840] The notification method and content of alert information are dynamically changed according to the user's emotional state.

[0841] For example, if the user is feeling stressed, the system will notify them in a quieter way, avoiding the audible alarm.

[0842] User Action

[0843] Step 1: Data monitoring

[0844] User:

[0845] Users can monitor analysis results and alert information in real time via their terminals.

[0846] Check your emotional state in conjunction with the alert information and calmly assess any abnormalities.

[0847] Step 2: Implementing countermeasures

[0848] User:

[0849] When implementing the displayed countermeasures, you can take into account your own emotional state.

[0850] For example, it helps a user in a stressed state to take calm action.

[0851] Step 3: Data feedback

[0852] User:

[0853] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[0854] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities next time.

[0855] Based on this detailed processing flow, a series of processes from collecting animal biological data to analysis, notification, and response are reliably carried out while taking into account the emotional state of the user.

[0856] Example 2

[0857] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0858] Conventional animal health and behavioral pattern monitoring systems only detect abnormalities in animals and do not provide a notification method that takes into account the user's emotional state. As a result, if the user is feeling stressed, the notification may not be received properly, making it difficult to respond quickly. Furthermore, there is no way for the user to understand their own emotional state and deal with the situation calmly.

[0859] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0860] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI to detect changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, means for analyzing the user's emotional state, and means for dynamically changing the alert notification method based on the user's emotional state. This enables efficient monitoring of the animal's health condition while enabling prompt and appropriate responses by using a notification method that takes the user's emotional state into consideration.

[0861] "Sensing device" refers to a device worn by an animal to collect biological data in real time.

[0862] "Biological Data" refers to information including physiological and behavioral data of an animal, such as the animal's heart rate, body temperature, and location.

[0863] "Video data" refers to video information of animal behavior and environment collected using a camera or other image capture device.

[0864] "Generative AI" refers to an artificial intelligence model for analyzing ecological and video data. This model has the ability to automatically analyze animal health and behavioral patterns.

[0865] A "database" refers to a data management system for storing analysis results and collected data, which allows for long-term data accumulation.

[0866] An "alert" is a notification that is generated when an abnormality is detected. This notification notifies the user that an abnormality has occurred and prompts them to take appropriate action.

[0867] "User emotional state" refers to data that indicates the psychological state and stress level of the user using the system.

[0868] "Emotion engine" refers to a system component that has the function of collecting and analyzing the user's emotional state.

[0869] A "dashboard" refers to a user interface that allows users to visually check analysis results and their own emotional state.

[0870] "Dynamic change of notification method" refers to changing the notification method and content of alert information in real time depending on the user's emotional state. Examples include email notifications and on-screen pop-up notifications.

[0871] Specific embodiments of the present invention are described below.

[0872] This system monitors the health and behavioral patterns of animals in real time and provides appropriate care. In particular, by combining it with an emotion engine, it can recognize the user's emotional state and dynamically change the appropriate notification method. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine.

[0873] The server serves as the central hub of the entire system and performs the following processes: First, the server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals. These sensing devices can be ordinary heart rate monitors or temperature sensors. The server also streams video data from cameras installed in the animal enclosures and stores it in a temporary storage area. This video data is used to analyze the behavioral patterns of the animals.

[0874] The server then cleans the collected ecology data, correcting and deleting inaccurate or missing data. It also removes noise from the video data and performs frame correction to improve the accuracy of the analysis. Generative AI is then used to analyze the ecology data and video data. This generative AI detects abnormal values ​​in the animal's heart rate and body temperature and identifies behavioral patterns.

[0875] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. For example, if a lion's heart rate suddenly rises, the server will interpret this as a sign of stress or illness.

[0876] The server also collects the user's emotional state through the user interface. The emotion engine evaluates the user's stress level based on the user's operation speed and response time. Based on this information, the server analyzes the user's emotional state and dynamically changes the notification method for alert information. For example, if the user is feeling stressed, the server can change the emergency alert notification to an email notification, avoiding the need for an immediate response.

[0877] The analysis results are stored in a database, allowing data to be accumulated over a long period of time. This allows for long-term monitoring of the animal's health. If an abnormality is detected, the server generates an alert and sends the alert information using a notification method that takes the user's emotional state into consideration.

[0878] The terminal is the interface through which users use the system, receiving analysis results and alert information from the server in real time. To visually display the received data, the terminal generates graphs and charts and provides information to the user. Furthermore, the terminal displays the user's emotional state, which is visualized together with the analysis results, helping the user to calmly determine whether an abnormality has occurred. The terminal also appropriately manages the notification method and content of alert information according to the user's emotional state.

[0879] Finally, users can use the system to manage their animals' health and behavioral patterns. They can monitor analysis results and alert information in real time, and calmly assess abnormalities while checking their own emotional state. Furthermore, users can provide detailed feedback on abnormal behavior and health conditions to improve the analysis accuracy of the generated AI.

[0880] Examples of prompts include:

[0881] "Lion's heart rate is significantly higher than normal. If this abnormality persists, immediate medical attention is required. Due to the user's high stress levels, a notification has been sent via email."

[0882] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0883] Step 1:

[0884] Data collection

[0885] Server operation: The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals, as well as video data from cameras installed in the animal enclosures.

[0886] Input: Biological data from sensing devices and video data from cameras.

[0887] Output: Biological data and video data stored in temporary storage area.

[0888] Specific operation: The server collects data from the sensing device every 5 seconds and stores it in a temporary storage area. It also streams camera footage and stores it in the temporary storage area.

[0889] Step 2:

[0890] Data Preprocessing

[0891] Server operation: The server cleans the collected biological data, corrects and deletes inaccurate and missing data, and removes noise from video data and performs frame correction.

[0892] Input: Biological data and video data in temporary storage area.

[0893] Output: Preprocessed and clean data.

[0894] Specific operation: The server executes a data cleaning algorithm to remove inaccurate points in the biological data and applies a noise reduction filter to the video data.

[0895] Step 3:

[0896] Data analysis

[0897] Server operation: The server uses generative AI to analyze the ecology and video data, specifically detecting abnormalities in heart rate and body temperature and identifying animal behavior patterns.

[0898] Input: Preprocessed ecological data and video data.

[0899] Output: Anomaly detection report and specific information on behavioral patterns as analysis results.

[0900] Specific operation: The generative AI model is executed to generate alert data when the heart rate exceeds a certain range or when abnormal movement is detected from the video.

[0901] Step 4:

[0902] Emotion data collection and analysis

[0903] Server operation: The server uses an emotion engine to collect and analyze user emotion data. It monitors the user's operation speed and response time to evaluate their stress level.

[0904] Input: User operation data and response data.

[0905] Output: Parsed user emotional state data.

[0906] Specific operation: The emotion engine analyzes the log data of the user interface and quantifies the user's stress level.

[0907] Step 5:

[0908] Data Retention and Notification

[0909] Server operation: The server stores the analysis results in a database and generates an alert if an anomaly is detected. The alert dynamically changes the notification method taking into account the user's emotional state.

[0910] Input: Anomaly detection report, emotional state data.

[0911] Output: Data stored in the database and alert information sent.

[0912] Specific operation: The server saves the anomaly detection report in a database, and based on the analysis results of the emotion engine, selects an appropriate notification method, such as email or a pop-up notification on the screen, and sends an alert to the user.

[0913] Step 6:

[0914] User Interface Operations

[0915] Device operation: The device receives analysis results and alert information from the server in real time and visually displays them, as well as the user's emotional state.

[0916] Input: Analysis results and alert information from the server.

[0917] Output: Visualized analysis results, alert information, and user emotional state.

[0918] Specific operation: The device converts the received data into graphs and charts and displays them, showing the user's emotional state in real time.

[0919] Step 7:

[0920] Data Feedback

[0921] User Action: The user uses the system to monitor the health and behavioral patterns of the animals and provides detailed feedback on abnormal behavior and health status.

[0922] Input: Analysis results and alert information.

[0923] Output: Feedback data from users.

[0924] Specific operation: The user monitors the system and, if an abnormality is detected, takes countermeasures and inputs the results into the system as feedback.

[0925] (Application example 2)

[0926] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0927] Currently, there are systems that monitor the health and behavioral patterns of animals in real time, but they do not take into account the user's emotional state. This results in inappropriate notifications being sent when the user feels stressed, resulting in an inability to respond appropriately. Furthermore, there is a lack of technology that can effectively analyze biological data from sensors and camera footage to quickly detect abnormalities. Therefore, there is a need for flexible notifications that take into account the user's emotional state while effectively managing animal health.

[0928] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting animal ecology data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecology data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, and means for dynamically changing the notification method based on the user's emotional state using an emotion engine that collects and analyzes the user's emotional data. This enables effective monitoring of the animal's health condition and flexible notification that takes the user's emotional state into consideration.

[0929] A "sensing device" is a device used to collect biological and operational data of animals and machines in real time.

[0930] "Video data" means visually recorded data of animal or machine behavior obtained by a camera or other visual sensor.

[0931] "Generative AI" is a technology that uses artificial intelligence techniques to analyze large amounts of data and detect abnormalities in health conditions and changes in behavioral patterns.

[0932] The "database" is a system for storing and managing analysis results, ecological data, and video data over the long term.

[0933] An "alert" is a warning message or signal that notifies the user when an abnormality is detected.

[0934] An "emotion engine" is a technology for analyzing a user's emotional state and assessing the user's emotions based on collected data.

[0935] A "prompt sentence" is a sentence used to explain to the user specific analysis results or abnormal behavior generated by the generative AI.

[0936] The "dashboard" is an interface that allows users to visually check, monitor, and manage analysis results and emotional states.

[0937] "Notification method" refers to the means by which alerts and information are communicated to users, such as email or instant messaging.

[0938] This invention provides a system that monitors the health status and behavior patterns of animals or machines in real time and provides effective countermeasures taking into account the emotional state of the user. The system includes a sensing device, a camera, a generative AI, an emotion engine, a database, a dashboard, and a notification means.

[0939] System Configuration

[0940] 1. Sensing Devices

[0941] This device is attached to animals or machines and collects biological and behavioral data such as heart rate, body temperature, and vibration in real time. The collected data is sent to a server.

[0942] 2. Camera

[0943] This device captures the target's movement patterns as video data, which is then sent to a server for analysis.

[0944] 3. Server

[0945] It is the central part of the entire system and performs the following processes:

[0946] Data collection from sensing devices and cameras.

[0947] Pre-processing of collected data (noise removal, data correction).

[0948] Data analysis using generative AI.

[0949] Generate alerts when anomalies are detected.

[0950] Analyzing user emotional states using an emotion engine.

[0951] Analysis results and alert information are stored in a database and sent to users.

[0952] Program processing explanation

[0953] 1. Data Collection

[0954] The server receives real-time data such as heart rate, body temperature, and vibration from sensing devices attached to animals and machines. It also receives video data from cameras installed in the breeding area and factory. This data is stored in a temporary storage area.

[0955] 2. Data Preprocessing

[0956] The server cleanses the collected data, correcting and deleting inaccurate or missing data, and also removes noise from video data and performs frame correction.

[0957] 3. Data Analysis

[0958] The server uses generative AI to analyze the biological data and video data. For example, if the temperature sensor data suddenly rises, it may indicate that the machine is overheating. This allows the server to detect changes in the health and behavior patterns of animals and machines.

[0959] 4. Emotional Data Collection and Analysis

[0960] The emotion engine collects user emotional data and analyzes their emotional state, allowing the system to take appropriate measures such as changing notification methods if the user is feeling stressed.

[0961] 5. Data Retention and Notification

[0962] The server stores the analysis results in a database and generates an alert if an abnormality is detected. The emotion engine takes the user's emotional state into account and dynamically changes the notification method and content. For example, if the user is feeling stressed, they can select to be notified by email.

[0963] Specific examples

[0964] In a factory application, the operating status of a robot is monitored. Data from vibration sensors and video data from cameras are sent to a server from sensing devices. A generative AI model analyzes this data and detects anomalies. An emotion engine evaluates the user's emotional state and selects an email notification if, for example, the user is experiencing high stress.

[0965] An example prompt is:

[0966] Sensor data: Temperature is over 80 degrees.

[0967] Camera footage analysis: Robot movement is unstable.

[0968] Emotion Engine Input: The user is concerned about the high temperature data from the sensor and the results of the video analysis.

[0969] This system allows for efficient health management of animals or machines, while also providing flexible responses that take into account the emotional state of the user.

[0970] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0971] Step 1: Data collection

[0972] The server receives biological data such as heart rate, body temperature, and vibration in real time from sensing devices attached to animals and machines. It also receives video data from cameras of the breeding areas and factory interiors. The input is numerical data of heart rate and body temperature and video data, and the output is raw data stored in a temporary storage area. Specifically, it retrieves data from sensors and cameras using HTTP requests.

[0973] Step 2: Preprocessing the data

[0974] The server cleans the collected biological data and video data, correcting and deleting inaccurate and missing data. It also performs noise removal and frame correction on video data. The input is raw data stored in a temporary storage area, and the output is cleaned, consistent data. Specific operations include applying data cleaning algorithms and noise removal filters.

[0975] Step 3: Data analysis

[0976] The server uses the generative AI model to analyze the cleaned biological data and video data. This detects abnormal values ​​in heart rate and body temperature and identifies changes in behavior patterns. The input is the pre-processed data, and the output is the analysis results. Specifically, the data is input into the generative AI model and the analysis results are obtained.

[0977] Step 4: Collect and analyze emotion data

[0978] The server uses an emotion engine to collect and analyze the user's emotional data. It evaluates the user's emotional state based on the text data and physical condition data entered into the system. The input is the user's text data and physical condition data, and the output is the analysis result of the user's emotional state. Specifically, the text and physical condition data are entered into the emotion engine, and an emotion evaluation result is obtained.

[0979] Step 5: Alerting and Notification

[0980] The server generates an alert if an anomaly is detected based on the results of data analysis and sentiment analysis. Furthermore, it uses an emotion engine to select a notification method that takes into account the user's emotional state, and notifies the user of the alert information. The input is the results of data analysis and sentiment analysis, and the output is the alert information sent to the user. Specifically, the server uses an anomaly detection algorithm to generate an alert and sends the notification via the selected method (email or instant message).

[0981] Step 6: Data storage and feedback

[0982] The server stores the analysis results and alert information in a database. Users can check the displayed information and provide feedback to help improve the accuracy of the generative AI model and emotion engine. The input is the analysis results and user feedback, and the output is an updated database. Specific operations include writing the analysis results and integrating the feedback data.

[0983] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0984] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0985] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0986] [Third embodiment]

[0987] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0988] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0989] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0990] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0991] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0992] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0993] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0994] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0995] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0996] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0997] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0998] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0999] System Overview

[1000] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[1001] Program Description

[1002] Server Processing

[1003] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[1004] 1. Data Collection

[1005] The server receives real-time biological data such as heart rate, body temperature, and location information from the sensing devices attached to the animals, allowing the server to instantly grasp the animal's condition.

[1006] The server receives video data from cameras installed in the breeding area and captures the behavior of the animals.

[1007] 2. Data Analysis

[1008] The collected ecological and video data is analyzed by generative AI, which uses this data to analyze the animals' health and behavioral patterns and detect any abnormalities.

[1009] For example, if the AI ​​detects an abnormally elevated heart rate, it will determine whether this is a sign of stress or illness.

[1010] 3. Data Retention and Notification

[1011] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[1012] If an abnormality is detected, the server immediately generates an alert and notifies the user. Alerts are classified according to the type and urgency of the abnormality.

[1013] Terminal handling

[1014] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[1015] 1. Receiving and displaying data

[1016] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[1017] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[1018] 2. Managing Notifications

[1019] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[1020] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[1021] User Action

[1022] Users use the system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[1023] 1. Data Monitoring

[1024] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[1025] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[1026] 2. Data Management and Feedback

[1027] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[1028] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[1029] Specific examples

[1030] A specific example is a health management system for lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action.

[1031] This system allows for efficient monitoring of animal health, early detection of abnormalities, and rapid response, thereby improving animal welfare and optimizing the breeding environment.

[1032] The processing flow will be explained below.

[1033] Server Processing

[1034] Step 1: Data collection

[1035] server:

[1036] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[1037] The server receives video data from cameras installed in the breeding area.

[1038] The collected data is stored in a temporary storage area.

[1039] Step 2: Preprocessing the data

[1040] server:

[1041] The server cleans the collected biological data, correcting and deleting inaccurate or missing data.

[1042] Noise removal and frame correction are also performed on the video data.

[1043] Step 3: Data analysis

[1044] server:

[1045] The server uses generative AI to analyze biological data in real time, detecting abnormalities in heart rate and body temperature.

[1046] The server uses video analysis tools to analyze the animal's behavioral patterns and detect abnormal behavior.

[1047] Step 4: Save Data

[1048] server:

[1049] The analysis results are stored in a database, which allows data to be accumulated over a long period of time.

[1050] The data will be organized chronologically to allow for quick searching and reference.

[1051] Step 5: Alerting and Notification

[1052] server:

[1053] If an anomaly is detected, the server immediately generates an alert, which includes the type of anomaly and its urgency.

[1054] Sends generated alerts to the device.

[1055] Terminal handling

[1056] Step 1: Receiving Data

[1057] Device:

[1058] The terminal receives analysis results and alert information from the server in real time.

[1059] Step 2: Visualize the data

[1060] Device:

[1061] Generate graphs and charts to visually display the received data, such as changes in heart rate or body temperature.

[1062] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[1063] Step 3: Sending notifications

[1064] Device:

[1065] Alert information is notified to the user via push notification, email, etc.

[1066] User Action

[1067] Step 1: Data monitoring

[1068] User:

[1069] Users can monitor analysis results and alert information in real time via their terminals.

[1070] If an important alert occurs, immediately check the notification and take the necessary action.

[1071] Step 2: Implementing countermeasures

[1072] User:

[1073] Based on the countermeasures presented by the server, appropriate care is provided for the animal's health condition and behavioral patterns.

[1074] For example, if an abnormality is detected, a veterinarian will be contacted and a detailed health check will be requested.

[1075] Step 3: Data feedback

[1076] User:

[1077] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[1078] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities in the future.

[1079] In this way, the entire process from collecting animal ecological data to analysis, notification, and response is carried out reliably.

[1080] Example 1

[1081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1082] Modern animal husbandry environments require effective systems for real-time monitoring of animal health and behavioral patterns and early detection of abnormalities. Conventional methods have fragmented processes for data collection, analysis, and notification, making it difficult to respond immediately. They also lack an interface that allows users to intuitively grasp the health status of animals. To solve these problems and improve animal welfare, an integrated system must be built.

[1083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1084] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for sending the generated alert to a terminal and the terminal notifying the user of it in real time, means for visually displaying the analysis results through a user interface and visualizing them as graphs or charts, and means for the user to refer to past data and analyze trends. This makes it possible to efficiently monitor the animal's health condition and behavioral patterns, detect abnormalities early, and take appropriate measures promptly.

[1085] A "sensing device" is a device that is attached to an animal and collects biological data such as heart rate, body temperature, and location information in real time.

[1086] "Video data" refers to digital data that records the movements and behavior of animals, captured using a video device such as a camera.

[1087] "Generative AI" is an artificial intelligence technology that analyzes ecological and video data to detect changes in animals' health and behavioral patterns.

[1088] A "database" is an information storage system for saving analysis results, and provides long-term data accumulation and backup functions.

[1089] An "alert" is a warning message that is generated when an abnormality is detected, and is a means of notifying the user.

[1090] A "terminal" is an interface device that allows users to use the system and receives and displays alerts and analysis results from the server.

[1091] A "user interface" is a collection of screens and functions that contain visual and operational elements that allow a user to interact with a system.

[1092] A "graph" is a diagram for visually displaying data, visualizing fluctuations along a time axis.

[1093] A "chart" is a graphic representation of numerical data visually, and includes formats such as bar graphs and line graphs.

[1094] A "trend" refers to a tendency or tendency observed from past data.

[1095] System Overview

[1096] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[1097] Program Description

[1098] Server Processing

[1099] The server is the center of this system, and performs data collection, analysis, storage, and notification. Specifically, it performs the following processes:

[1100] 1. Data Collection

[1101] The server receives real-time biological data, such as heart rate, body temperature, and location information, from sensing devices attached to the animals. For example, it receives data that a lion's heart rate has increased from 60 bpm to 80 bpm.

[1102] The server receives video data from cameras installed in the enclosure, capturing footage of the lions' movements and postures every 10 seconds.

[1103] 2. Data Transmission

[1104] The server simultaneously receives the collected biological data and video data, so it needs to maintain stable data communication over the network. For example, the server synchronizes data with the device using Wi-Fi.

[1105] 3. Data Analysis

[1106] The generative AI model on the server analyzes the received biological data and video data in real time. Specifically, the AI ​​detects abnormal heart rate patterns and behavioral abnormalities based on the video data. For example, it detects stress by linking a sudden increase in heart rate with vigorous movements in the video data.

[1107] 4. Data storage and alert generation

[1108] The server stores the results of the analysis by the generative AI in a database. For example, the analysis results are stored in a database as a lion's health log, including heart rate, body temperature, and behavioral patterns.

[1109] The server generates an alert if it detects an abnormality. For example, if the heart rate is abnormally high, the server adds "Abnormal Heart Rate: Urgent" to the alert list.

[1110] 5. User Notices

[1111] The server sends the generated alerts to the device. For example, when a new alert is generated, the server notifies the user by push notification or email.

[1112] The device notifies the user of the received alert in real time. For example, the device may use the smartphone's push notification function to send the user a message saying, "The lion's heart rate is abnormal. Please check immediately."

[1113] Terminal handling

[1114] The terminal is an interface that allows users to use the system. Specifically, it includes the following processes:

[1115] 1. Receiving and displaying data

[1116] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[1117] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[1118] 2. Managing Notifications

[1119] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[1120] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[1121] User Action

[1122] Users use the system to manage the health and behavioral patterns of their animals, including the following processes:

[1123] 1. Data Monitoring

[1124] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[1125] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[1126] 2. Data Management and Feedback

[1127] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[1128] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[1129] Specific examples

[1130] A specific example is a health management system for lions at a zoo. The server receives heart rate and body temperature data in real time from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action. This system makes it possible to efficiently monitor the health of animals, detect abnormalities early, and respond quickly. This improves animal welfare and optimizes the captive environment.

[1131] Prompt Sentence Examples

[1132] "Analyze the heart rate and temperature data received from the sensing device attached to the lion, as well as the camera footage, to assess the lion's health and behavioral patterns. If you detect any abnormalities, state the reason and recommend a course of action."

[1133] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1134] Step 1:

[1135] Data collection

[1136] The server receives biological data such as heart rate, body temperature, and location information in real time from sensing devices attached to the animals. The input is data from various sensors, and the output is raw data stored in the server. Specifically, the server receives and temporarily stores data on heart rate increases from 60 bpm to 80 bpm and body temperature data every unit of time.

[1137] Step 2:

[1138] Video data collection

[1139] The server receives video data from cameras installed in the enclosure. The input is the video signal from the camera, and the output is the video data stored in the server. For example, the server captures and stores video of lions' movements every 10 seconds.

[1140] Step 3:

[1141] Data transmission

[1142] The server communicates via a network to stably receive the collected biological data and video data. The input is data from the sensing device and camera, and the output is the integrated data sent to the server. Specifically, the server synchronizes the real-time data using Wi-Fi.

[1143] Step 4:

[1144] Data analysis

[1145] The generative AI model on the server analyzes the received ecological data and video data in real time. The input is the ecological data and video data stored on the server, and the output is the analyzed health status and behavioral pattern changes. Specifically, the AI ​​analyzes sudden increases in heart rate and violent animal movements based on the video to detect stress or health abnormalities.

[1146] Step 5:

[1147] Data storage

[1148] The server stores the results of the analysis by the generative AI in a database. The input is the analyzed health data and behavioral pattern data, and the output is the historical data stored in the database. Specifically, the server adds logs of the lion's heart rate, body temperature, and behavioral patterns to the database.

[1149] Step 6:

[1150] Alert Generation

[1151] The server generates an alert if it detects an abnormality. The input is the abnormal data from the generation AI, and the output is a warning message added to the alert list. For example, if the heart rate is abnormally high, the server adds "Heart rate abnormal: emergency" to the alert list.

[1152] Step 7:

[1153] Alert Notification

[1154] The server sends the generated alert to the device. The input is the alert message, and the output is the alert notification sent to the device. Specifically, the server sends an emergency message to the user via push notification or email.

[1155] Step 8:

[1156] Real-time notifications

[1157] The terminal notifies the user of the received alert in real time. The input is the alert notification sent from the server, and the output is a warning message displayed on the user's terminal. For example, the terminal may display a message such as "The lion's heart rate is abnormal. Please check immediately."

[1158] Step 9:

[1159] Data Monitoring

[1160] The user uses a terminal to monitor the analysis results and alert information from the server in real time. The input is the analysis results displayed on the terminal's dashboard, and the output is the user's judgment and response. Specifically, the user looks at the graphs and charts on the terminal to identify abnormalities.

[1161] Step 10:

[1162] Data Management and Feedback

[1163] Users can view long-term data to understand trends in animal health and behavior. The input is historical database information, and the output is analysis results and feedback. Specifically, users can view a year's worth of health data in graphs and provide feedback to the system to improve the accuracy of the generative AI.

[1164] (Application example 1)

[1165] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1166] In recent years, as the complexity and operating hours of robots used in factories have increased, it has become increasingly important to monitor the status of robots in real time and detect abnormalities early. However, current systems lack a mechanism for immediately detecting and efficiently notifying robot abnormalities, which is a factor that affects production efficiency and safety. A solution to this issue is needed.

[1167] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1168] In this invention, the server includes means for collecting operational data of the object from the sensing device in real time, means for receiving video data and analyzing the work pattern of the object, means for analyzing the operational data and video data using a generation AI and detecting changes in the operating state and work pattern of the object, means for saving the analysis results in a database and generating an alert when an abnormality is detected, and means for notifying engineers of the analysis results and alert information. This makes it possible to monitor robot abnormalities in real time and to immediately notify engineers when an abnormality is detected.

[1169] A "sensing device" is a device that collects operational data and environmental data (temperature, vibration, power consumption, etc.) of an object in real time.

[1170] "Operation data" refers to data on the status of an object while it is in operation (such as temperature, vibration, and power consumption).

[1171] "Video data" refers to video information of an object being worked on, captured using photographic equipment such as a camera.

[1172] "Work pattern" refers to the pattern of actions and behaviors of an object when performing a specific task or operation.

[1173] "Generative AI" is artificial intelligence that analyzes changes in the operating status and work patterns of objects based on collected operational data and video data.

[1174] "Analysis results" refer to the information and insights obtained when operational data and video data are analyzed by the generating AI.

[1175] A "database" is a storage medium or system for long-term storage and management of analysis results and ecological data.

[1176] An "alert" is a warning notification that is generated when an abnormality is detected and is immediately reported to a technician.

[1177] An "engineer" is someone who receives analysis results and alert information and takes action.

[1178] A "dashboard" is a user interface that allows engineers to visually check analysis results and alert information.

[1179] System Overview

[1180] This invention is a system that enables preventive maintenance by monitoring the status and operation patterns of factory robots in real time and immediately notifying engineers if an abnormality is detected. The system mainly consists of a sensing device, a video analysis tool, a generative AI, and a user interface with a dashboard.

[1181] Program Description

[1182] Server Processing

[1183] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[1184] 1. Data Collection

[1185] The server receives real-time operational data such as temperature, vibration, and power consumption from sensing devices attached to factory robots, allowing the status of the target object to be grasped immediately.

[1186] The server receives video data from cameras installed in the factory and captures the robot's work patterns.

[1187] 2. Data Analysis

[1188] The collected operational and video data is analyzed by the generative AI, which uses this data to analyze the operating status and working patterns of the object and detect any abnormalities.

[1189] For example, if the AI ​​detects an abnormal rise in temperature, it determines whether this is a sign of overheating or abnormal operation.

[1190] 3. Data Retention and Notification

[1191] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[1192] If an anomaly is detected, the server immediately generates an alert and notifies the technician. Alerts are categorized according to the type of anomaly and its urgency.

[1193] Terminal handling

[1194] The terminal is the interface through which the engineer uses the system. The process is explained below in natural language.

[1195] 1. Receiving and displaying data

[1196] The terminal receives analysis results and alert information from the server, allowing engineers to understand the status of the target object in real time.

[1197] The data is displayed visually and visualized on a dashboard as graphs and charts, for example showing temperature or vibration fluctuations over time, allowing engineers to spot anomalies at a glance.

[1198] 2. Managing Notifications

[1199] The device instantly notifies technicians of important alert information in a variety of ways, including push notifications and emails.

[1200] If an abnormality is detected, countermeasures are presented on the terminal, helping the technician to take appropriate measures immediately.

[1201] User Action

[1202] The user, as an engineer, uses the system to manage the operating status and work patterns of the factory robots. The process is explained below in natural language.

[1203] 1. Data Monitoring

[1204] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[1205] For example, if a user is monitoring the temperature status of a factory robot, if an abnormal temperature rise or vibration pattern is detected, the system will notify them and they can quickly take inspection or repair measures.

[1206] 2. Data Management and Feedback

[1207] Users can view long-term data to understand trends in the operating status and work patterns of their equipment, allowing them to provide preventative maintenance.

[1208] If necessary, specific abnormal behaviors or conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[1209] Specific examples

[1210] A specific example is a robot maintenance system in a factory. The server receives temperature and vibration data in real time from sensing devices attached to the robot, and also captures camera footage. The generative AI analyzes this data and monitors the robot's operating status. If an abnormality is detected, an alert is generated and notified to the technician via the device. The technician can immediately analyze the cause of the abnormality and take appropriate action.

[1211] This system makes it possible to efficiently monitor the operating status of factory robots, detect abnormalities early, and respond quickly, thereby improving production efficiency and safety within the factory.

[1212] Prompt Sentence Examples

[1213] "The temperature of a factory robot is rising abnormally. An 85°C warning has been issued. Please provide instructions for generating an immediate alert and notifying a technician."

[1214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1215] Step 1:

[1216] The server collects operational data such as temperature, vibration, and power consumption in real time from sensing devices attached to factory robots. This input data allows the robot's status to be constantly monitored. The collected data is initially processed on the server and converted into an appropriate format.

[1217] Step 2:

[1218] The server receives video data in real time from cameras installed in the factory. The video data is used to visually capture the robot's movements and work patterns. The received video is divided into frames and saved.

[1219] Step 3:

[1220] The server provides collected operational and video data as input to the generative AI model, which analyzes this data to detect the object's operating status and work patterns. If an abnormality is detected, the AI ​​model will recognize signs such as elevated temperatures or abnormal vibrations.

[1221] Step 4:

[1222] The server stores the analysis results of the generative AI model in a database. The stored data is organized with time information to facilitate future searches and analysis, while maintaining the integrity and security of the data.

[1223] Step 5:

[1224] If the generative AI model detects an anomaly, the server immediately generates an alert, generating an appropriate message depending on the type and urgency of the anomaly, which is then recorded in the alert log.

[1225] Step 6:

[1226] The server notifies the technician of the generated alert. The notification can be sent in the form of a push notification, email, SMS, etc. For example, in the case of a temperature abnormality, a message will be sent saying, "Robot A's temperature has reached 85°C. Immediate action is required."

[1227] Step 7:

[1228] The terminal visually displays the analysis results and alert information received from the server. The results are converted into graphs and charts on the user interface, allowing engineers to check them in real time. For example, fluctuations in temperature or vibration can be seen at a glance.

[1229] Step 8:

[1230] Users use their terminals to monitor and check the analysis results and alert information from the server. When an abnormality is detected, users can provide feedback to the system to take immediate action. Specifically, users can record actions such as "Check that Robot A's temperature is abnormal and check the cooling system."

[1231] Step 9:

[1232] Users can refer to long-term data to understand trends in the operating status and work patterns of their equipment, enabling them to plan preventative maintenance and improvements, thereby improving production efficiency and safety within their factories.

[1233] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1234] System Overview

[1235] This invention is a system for monitoring the health status and behavioral patterns of animals in real time and providing appropriate care. In particular, by combining it with an emotion engine, it is possible to recognize the user's emotional state and dynamically change the notification method and content of alert information based on that. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine, and aims to improve animal welfare and optimize the housing environment.

[1236] Program Description

[1237] Server Processing

[1238] The server is the central hub of the entire system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[1239] 1. Data Collection

[1240] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[1241] The server receives video data from cameras installed in the breeding area.

[1242] Both data are stored in a temporary storage area.

[1243] 2. Data Preprocessing

[1244] The server cleans the collected biometric data and corrects and deletes inaccurate or missing data.

[1245] Noise is also removed from the video data, and frame correction is performed.

[1246] 3. Data Analysis

[1247] The server uses generative AI to analyze the biological and video data, detecting abnormalities in heart rate and body temperature and identifying behavioral patterns.

[1248] For example, if a lion's heart rate suddenly increases, it may be interpreted as a sign of stress or illness.

[1249] 4. Emotional Data Collection and Analysis

[1250] The server uses an emotion engine to collect the user's emotion data and analyze the user's emotional state.

[1251] For example, if the user is feeling stressed, the method of notifying the alert information is changed.

[1252] 5. Data Retention and Notification

[1253] The analysis results are stored in a database, allowing data to be accumulated over a long period of time.

[1254] If an abnormality is detected, the server generates an alert and sends it using a notification method that takes into account the user's emotional state.

[1255] Terminal handling

[1256] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[1257] 1. Receiving and displaying data

[1258] The terminal receives analysis results and alert information from the server in real time.

[1259] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[1260] 2. Displaying the user's emotional state

[1261] The device displays the user's emotional state and visualizes it along with the analysis results. By checking their own emotional state, users can make calm decisions.

[1262] 3. Managing Notifications

[1263] The notification method and content of alert information are dynamically changed depending on the user's emotional state. For example, if the user is feeling stressed, the notification may be changed to email, rather than prompting an immediate response.

[1264] User Action

[1265] Users can use this system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[1266] 1. Data Monitoring

[1267] Users can monitor analysis results and alert information in real time via their terminals.

[1268] The user calmly judges whether there is anything abnormal while checking their own emotional state.

[1269] 2. Implementing emotional coping strategies

[1270] When implementing the displayed countermeasures, the user can take measures that take into account his or her own emotional state.

[1271] For example, it helps a user in a stressed state to take calm action.

[1272] 3. Data Feedback

[1273] Users can provide detailed feedback on abnormal behavior and health conditions to improve the analytical accuracy of the generative AI.

[1274] For example, by recording the progress after treatment and using this information as a reference for the next abnormality detection, response times can be shortened.

[1275] Specific examples

[1276] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. The generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, the emotion engine recognizes the user's emotional state and changes the notification method for alert information. For example, if the user is feeling stressed, an email notification can be sent to avoid immediate response. This system makes it possible to efficiently monitor the health of animals while providing care that takes the user's emotional state into consideration.

[1277] The processing flow will be explained below.

[1278] Server Processing

[1279] Step 1: Collecting data from sensing devices

[1280] server:

[1281] The server receives real-time biological data such as heart rate, body temperature, and location information from sensing devices attached to the animals.

[1282] The collected data is stored in a temporary storage area.

[1283] Step 2: Collecting video data from the camera

[1284] server:

[1285] The server receives video data in real time from cameras installed within the breeding area.

[1286] Video data is also stored in the temporary storage area.

[1287] Step 3: Preprocessing the data

[1288] server:

[1289] The server cleans the biometric data, correcting and deleting inaccurate or missing data.

[1290] Noise removal and frame correction are also performed on the video data.

[1291] Step 4: Analysis of ecological and video data

[1292] server:

[1293] The collected and pre-processed data is analyzed using generative AI to detect abnormalities in heart rate and body temperature.

[1294] Animal behavior patterns are extracted from video data and abnormal behavior is detected.

[1295] Step 5: Emotional Data Collection and Analysis

[1296] server:

[1297] The emotion engine collects emotion data from the user's device, including facial expression recognition and voice analysis.

[1298] The server analyzes the collected emotional data and determines the user's stress level and level of urgency.

[1299] Step 6: Data storage and alert generation

[1300] server:

[1301] The analysis results are stored in a database, and data is accumulated over a long period of time.

[1302] If an abnormality is detected, an alert is generated, and the alert information includes the type of abnormality and its urgency.

[1303] Step 7: Sending notifications

[1304] server:

[1305] A notification method for alert information is dynamically selected based on the emotional state of a user.

[1306] For example, if the user is feeling stressed, the notification method can be changed to email, and immediate action will not be urged.

[1307] Terminal handling

[1308] Step 1: Receiving Data

[1309] Device:

[1310] The terminal receives analysis results and alert information from the server in real time.

[1311] Step 2: Visualize the data

[1312] Device:

[1313] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[1314] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[1315] Step 3: Obtaining and displaying emotion data

[1316] Device:

[1317] The device analyzes and displays the user's emotional state in real time, using a camera and microphone to collect emotional data.

[1318] The user's current emotional state is displayed and visualized along with the analysis results.

[1319] Step 4: Manage notifications

[1320] Device:

[1321] The notification method and content of alert information are dynamically changed according to the user's emotional state.

[1322] For example, if the user is feeling stressed, the system will notify them in a quieter way, avoiding the audible alarm.

[1323] User Action

[1324] Step 1: Data monitoring

[1325] User:

[1326] Users can monitor analysis results and alert information in real time via their terminals.

[1327] Check your emotional state in conjunction with the alert information and calmly assess any abnormalities.

[1328] Step 2: Implementing countermeasures

[1329] User:

[1330] When implementing the displayed countermeasures, you can take into account your own emotional state.

[1331] For example, it helps a user in a stressed state to take calm action.

[1332] Step 3: Data feedback

[1333] User:

[1334] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[1335] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities next time.

[1336] Based on this detailed processing flow, a series of processes from collecting animal biological data to analysis, notification, and response are reliably carried out while taking into account the emotional state of the user.

[1337] Example 2

[1338] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1339] Conventional animal health and behavioral pattern monitoring systems only detect abnormalities in animals and do not provide a notification method that takes into account the user's emotional state. As a result, if the user is feeling stressed, the notification may not be received properly, making it difficult to respond quickly. Furthermore, there is no way for the user to understand their own emotional state and deal with the situation calmly.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1341] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI to detect changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, means for analyzing the user's emotional state, and means for dynamically changing the alert notification method based on the user's emotional state. This enables efficient monitoring of the animal's health condition while enabling prompt and appropriate responses by using a notification method that takes the user's emotional state into consideration.

[1342] "Sensing device" refers to a device worn by an animal to collect biological data in real time.

[1343] "Biological Data" refers to information including physiological and behavioral data of an animal, such as the animal's heart rate, body temperature, and location.

[1344] "Video data" refers to video information of animal behavior and environment collected using a camera or other image capture device.

[1345] "Generative AI" refers to an artificial intelligence model for analyzing ecological and video data. This model has the ability to automatically analyze animal health and behavioral patterns.

[1346] A "database" refers to a data management system for storing analysis results and collected data, which allows for long-term data accumulation.

[1347] An "alert" is a notification that is generated when an abnormality is detected. This notification notifies the user that an abnormality has occurred and prompts them to take appropriate action.

[1348] "User emotional state" refers to data that indicates the psychological state and stress level of the user using the system.

[1349] "Emotion engine" refers to a system component that has the function of collecting and analyzing the user's emotional state.

[1350] A "dashboard" refers to a user interface that allows users to visually check analysis results and their own emotional state.

[1351] "Dynamic change of notification method" refers to changing the notification method and content of alert information in real time depending on the user's emotional state. Examples include email notifications and on-screen pop-up notifications.

[1352] Specific embodiments of the present invention are described below.

[1353] This system monitors the health and behavioral patterns of animals in real time and provides appropriate care. In particular, by combining it with an emotion engine, it can recognize the user's emotional state and dynamically change the appropriate notification method. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine.

[1354] The server serves as the central hub of the entire system and performs the following processes: First, the server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals. These sensing devices can be ordinary heart rate monitors or temperature sensors. The server also streams video data from cameras installed in the animal enclosures and stores it in a temporary storage area. This video data is used to analyze the behavioral patterns of the animals.

[1355] The server then cleans the collected ecology data, correcting and deleting inaccurate or missing data. It also removes noise from the video data and performs frame correction to improve the accuracy of the analysis. Generative AI is then used to analyze the ecology data and video data. This generative AI detects abnormal values ​​in the animal's heart rate and body temperature and identifies behavioral patterns.

[1356] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. For example, if a lion's heart rate suddenly rises, the server will interpret this as a sign of stress or illness.

[1357] The server also collects the user's emotional state through the user interface. The emotion engine evaluates the user's stress level based on the user's operation speed and response time. Based on this information, the server analyzes the user's emotional state and dynamically changes the notification method for alert information. For example, if the user is feeling stressed, the server can change the emergency alert notification to an email notification, avoiding the need for an immediate response.

[1358] The analysis results are stored in a database, allowing data to be accumulated over a long period of time. This allows for long-term monitoring of the animal's health. If an abnormality is detected, the server generates an alert and sends the alert information using a notification method that takes the user's emotional state into consideration.

[1359] The terminal is the interface through which users use the system, receiving analysis results and alert information from the server in real time. To visually display the received data, the terminal generates graphs and charts and provides information to the user. Furthermore, the terminal displays the user's emotional state, which is visualized together with the analysis results, helping the user to calmly determine whether an abnormality has occurred. The terminal also appropriately manages the notification method and content of alert information according to the user's emotional state.

[1360] Finally, users can use the system to manage their animals' health and behavioral patterns. They can monitor analysis results and alert information in real time, and calmly assess abnormalities while checking their own emotional state. Furthermore, users can provide detailed feedback on abnormal behavior and health conditions to improve the analysis accuracy of the generated AI.

[1361] Examples of prompts include:

[1362] "Lion's heart rate is significantly higher than normal. If this abnormality persists, immediate medical attention is required. Due to the user's high stress levels, a notification has been sent via email."

[1363] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1364] Step 1:

[1365] Data collection

[1366] Server operation: The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals, as well as video data from cameras installed in the animal enclosures.

[1367] Input: Biological data from sensing devices and video data from cameras.

[1368] Output: Biological data and video data stored in temporary storage area.

[1369] Specific operation: The server collects data from the sensing device every 5 seconds and stores it in a temporary storage area. It also streams camera footage and stores it in the temporary storage area.

[1370] Step 2:

[1371] Data Preprocessing

[1372] Server operation: The server cleans the collected biological data, corrects and deletes inaccurate and missing data, and removes noise from video data and performs frame correction.

[1373] Input: Biological data and video data in temporary storage area.

[1374] Output: Preprocessed and clean data.

[1375] Specific operation: The server executes a data cleaning algorithm to remove inaccurate points in the biological data and applies a noise reduction filter to the video data.

[1376] Step 3:

[1377] Data analysis

[1378] Server operation: The server uses generative AI to analyze the ecology and video data, specifically detecting abnormalities in heart rate and body temperature and identifying animal behavior patterns.

[1379] Input: Preprocessed ecological data and video data.

[1380] Output: Anomaly detection report and specific information on behavioral patterns as analysis results.

[1381] Specific operation: The generative AI model is executed to generate alert data when the heart rate exceeds a certain range or when abnormal movement is detected from the video.

[1382] Step 4:

[1383] Emotion data collection and analysis

[1384] Server operation: The server uses an emotion engine to collect and analyze user emotion data. It monitors the user's operation speed and response time to evaluate their stress level.

[1385] Input: User operation data and response data.

[1386] Output: Parsed user emotional state data.

[1387] Specific operation: The emotion engine analyzes the log data of the user interface and quantifies the user's stress level.

[1388] Step 5:

[1389] Data Retention and Notification

[1390] Server operation: The server stores the analysis results in a database and generates an alert if an anomaly is detected. The alert dynamically changes the notification method taking into account the user's emotional state.

[1391] Input: Anomaly detection report, emotional state data.

[1392] Output: Data stored in the database and alert information sent.

[1393] Specific operation: The server saves the anomaly detection report in a database, and based on the analysis results of the emotion engine, selects an appropriate notification method, such as email or a pop-up notification on the screen, and sends an alert to the user.

[1394] Step 6:

[1395] User Interface Operations

[1396] Device operation: The device receives analysis results and alert information from the server in real time and visually displays them, as well as the user's emotional state.

[1397] Input: Analysis results and alert information from the server.

[1398] Output: Visualized analysis results, alert information, and user emotional state.

[1399] Specific operation: The device converts the received data into graphs and charts and displays them, showing the user's emotional state in real time.

[1400] Step 7:

[1401] Data Feedback

[1402] User Action: The user uses the system to monitor the health and behavioral patterns of the animals and provides detailed feedback on abnormal behavior and health status.

[1403] Input: Analysis results and alert information.

[1404] Output: Feedback data from users.

[1405] Specific operation: The user monitors the system and, if an abnormality is detected, takes countermeasures and inputs the results into the system as feedback.

[1406] (Application example 2)

[1407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1408] Currently, there are systems that monitor the health and behavioral patterns of animals in real time, but they do not take into account the user's emotional state. This results in inappropriate notifications being sent when the user feels stressed, resulting in an inability to respond appropriately. Furthermore, there is a lack of technology that can effectively analyze biological data from sensors and camera footage to quickly detect abnormalities. Therefore, there is a need for flexible notifications that take into account the user's emotional state while effectively managing animal health.

[1409] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting animal ecology data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecology data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, and means for dynamically changing the notification method based on the user's emotional state using an emotion engine that collects and analyzes the user's emotional data. This enables effective monitoring of the animal's health condition and flexible notification that takes the user's emotional state into consideration.

[1410] A "sensing device" is a device used to collect biological and operational data of animals and machines in real time.

[1411] "Video data" means visually recorded data of animal or machine behavior obtained by a camera or other visual sensor.

[1412] "Generative AI" is a technology that uses artificial intelligence techniques to analyze large amounts of data and detect abnormalities in health conditions and changes in behavioral patterns.

[1413] The "database" is a system for storing and managing analysis results, ecological data, and video data over the long term.

[1414] An "alert" is a warning message or signal that notifies the user when an abnormality is detected.

[1415] An "emotion engine" is a technology for analyzing a user's emotional state and assessing the user's emotions based on collected data.

[1416] A "prompt sentence" is a sentence used to explain to the user specific analysis results or abnormal behavior generated by the generative AI.

[1417] The "dashboard" is an interface that allows users to visually check, monitor, and manage analysis results and emotional states.

[1418] "Notification method" refers to the means by which alerts and information are communicated to users, such as email or instant messaging.

[1419] This invention provides a system that monitors the health status and behavior patterns of animals or machines in real time and provides effective countermeasures taking into account the emotional state of the user. The system includes a sensing device, a camera, a generative AI, an emotion engine, a database, a dashboard, and a notification means.

[1420] System Configuration

[1421] 1. Sensing Devices

[1422] This device is attached to animals or machines and collects biological and behavioral data such as heart rate, body temperature, and vibration in real time. The collected data is sent to a server.

[1423] 2. Camera

[1424] This device captures the target's movement patterns as video data, which is then sent to a server for analysis.

[1425] 3. Server

[1426] It is the central part of the entire system and performs the following processes:

[1427] Data collection from sensing devices and cameras.

[1428] Pre-processing of collected data (noise removal, data correction).

[1429] Data analysis using generative AI.

[1430] Generate alerts when anomalies are detected.

[1431] Analyzing user emotional states using an emotion engine.

[1432] Analysis results and alert information are stored in a database and sent to users.

[1433] Program processing explanation

[1434] 1. Data Collection

[1435] The server receives real-time data such as heart rate, body temperature, and vibration from sensing devices attached to animals and machines. It also receives video data from cameras installed in the breeding area and factory. This data is stored in a temporary storage area.

[1436] 2. Data Preprocessing

[1437] The server cleanses the collected data, correcting and deleting inaccurate or missing data, and also removes noise from video data and performs frame correction.

[1438] 3. Data Analysis

[1439] The server uses generative AI to analyze the biological data and video data. For example, if the temperature sensor data suddenly rises, it may indicate that the machine is overheating. This allows the server to detect changes in the health and behavior patterns of animals and machines.

[1440] 4. Emotional Data Collection and Analysis

[1441] The emotion engine collects user emotional data and analyzes their emotional state, allowing the system to take appropriate measures such as changing notification methods if the user is feeling stressed.

[1442] 5. Data Retention and Notification

[1443] The server stores the analysis results in a database and generates an alert if an abnormality is detected. The emotion engine takes the user's emotional state into account and dynamically changes the notification method and content. For example, if the user is feeling stressed, they can select to be notified by email.

[1444] Specific examples

[1445] In a factory application, the operating status of a robot is monitored. Data from vibration sensors and video data from cameras are sent to a server from sensing devices. A generative AI model analyzes this data and detects anomalies. An emotion engine evaluates the user's emotional state and selects an email notification if, for example, the user is experiencing high stress.

[1446] An example prompt is:

[1447] Sensor data: Temperature is over 80 degrees.

[1448] Camera footage analysis: Robot movement is unstable.

[1449] Emotion Engine Input: The user is concerned about the high temperature data from the sensor and the results of the video analysis.

[1450] This system allows for efficient health management of animals or machines, while also providing flexible responses that take into account the emotional state of the user.

[1451] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1452] Step 1: Data collection

[1453] The server receives biological data such as heart rate, body temperature, and vibration in real time from sensing devices attached to animals and machines. It also receives video data from cameras of the breeding areas and factory interiors. The input is numerical data of heart rate and body temperature and video data, and the output is raw data stored in a temporary storage area. Specifically, it retrieves data from sensors and cameras using HTTP requests.

[1454] Step 2: Preprocessing the data

[1455] The server cleans the collected biological data and video data, correcting and deleting inaccurate and missing data. It also performs noise removal and frame correction on video data. The input is raw data stored in a temporary storage area, and the output is cleaned, consistent data. Specific operations include applying data cleaning algorithms and noise removal filters.

[1456] Step 3: Data analysis

[1457] The server uses the generative AI model to analyze the cleaned biological data and video data. This detects abnormal values ​​in heart rate and body temperature and identifies changes in behavior patterns. The input is the pre-processed data, and the output is the analysis results. Specifically, the data is input into the generative AI model and the analysis results are obtained.

[1458] Step 4: Collect and analyze emotion data

[1459] The server uses an emotion engine to collect and analyze the user's emotional data. It evaluates the user's emotional state based on the text data and physical condition data entered into the system. The input is the user's text data and physical condition data, and the output is the analysis result of the user's emotional state. Specifically, the text and physical condition data are entered into the emotion engine, and an emotion evaluation result is obtained.

[1460] Step 5: Alerting and Notification

[1461] The server generates an alert if an anomaly is detected based on the results of data analysis and sentiment analysis. Furthermore, it uses an emotion engine to select a notification method that takes into account the user's emotional state, and notifies the user of the alert information. The input is the results of data analysis and sentiment analysis, and the output is the alert information sent to the user. Specifically, the server uses an anomaly detection algorithm to generate an alert and sends the notification via the selected method (email or instant message).

[1462] Step 6: Data storage and feedback

[1463] The server stores the analysis results and alert information in a database. Users can check the displayed information and provide feedback to help improve the accuracy of the generative AI model and emotion engine. The input is the analysis results and user feedback, and the output is an updated database. Specific operations include writing the analysis results and integrating the feedback data.

[1464] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1465] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1466] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1467] [Fourth embodiment]

[1468] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1469] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1470] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1471] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1472] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1473] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1474] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1475] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1476] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1477] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1478] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1479] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1480] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1481] System Overview

[1482] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[1483] Program Description

[1484] Server Processing

[1485] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[1486] 1. Data Collection

[1487] The server receives real-time biological data such as heart rate, body temperature, and location information from the sensing devices attached to the animals, allowing the server to instantly grasp the animal's condition.

[1488] The server receives video data from cameras installed in the breeding area and captures the behavior of the animals.

[1489] 2. Data Analysis

[1490] The collected ecological and video data is analyzed by generative AI, which uses this data to analyze the animals' health and behavioral patterns and detect any abnormalities.

[1491] For example, if the AI ​​detects an abnormally elevated heart rate, it will determine whether this is a sign of stress or illness.

[1492] 3. Data Retention and Notification

[1493] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[1494] If an abnormality is detected, the server immediately generates an alert and notifies the user. Alerts are classified according to the type and urgency of the abnormality.

[1495] Terminal handling

[1496] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[1497] 1. Receiving and displaying data

[1498] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[1499] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[1500] 2. Managing Notifications

[1501] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[1502] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[1503] User Action

[1504] Users use the system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[1505] 1. Data Monitoring

[1506] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[1507] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[1508] 2. Data Management and Feedback

[1509] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[1510] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[1511] Specific examples

[1512] A specific example is a health management system for lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action.

[1513] This system allows for efficient monitoring of animal health, early detection of abnormalities, and rapid response, thereby improving animal welfare and optimizing the breeding environment.

[1514] The processing flow will be explained below.

[1515] Server Processing

[1516] Step 1: Data collection

[1517] server:

[1518] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[1519] The server receives video data from cameras installed in the breeding area.

[1520] The collected data is stored in a temporary storage area.

[1521] Step 2: Preprocessing the data

[1522] server:

[1523] The server cleans the collected biological data, correcting and deleting inaccurate or missing data.

[1524] Noise removal and frame correction are also performed on the video data.

[1525] Step 3: Data analysis

[1526] server:

[1527] The server uses generative AI to analyze biological data in real time, detecting abnormalities in heart rate and body temperature.

[1528] The server uses video analysis tools to analyze the animal's behavioral patterns and detect abnormal behavior.

[1529] Step 4: Save Data

[1530] server:

[1531] The analysis results are stored in a database, which allows data to be accumulated over a long period of time.

[1532] The data will be organized chronologically to allow for quick searching and reference.

[1533] Step 5: Alerting and Notification

[1534] server:

[1535] If an anomaly is detected, the server immediately generates an alert, which includes the type of anomaly and its urgency.

[1536] Sends generated alerts to the device.

[1537] Terminal handling

[1538] Step 1: Receiving Data

[1539] Device:

[1540] The terminal receives analysis results and alert information from the server in real time.

[1541] Step 2: Visualize the data

[1542] Device:

[1543] Generate graphs and charts to visually display the received data, such as changes in heart rate or body temperature.

[1544] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[1545] Step 3: Sending notifications

[1546] Device:

[1547] Alert information is notified to the user via push notification, email, etc.

[1548] User Action

[1549] Step 1: Data monitoring

[1550] User:

[1551] Users can monitor analysis results and alert information in real time via their terminals.

[1552] If an important alert occurs, immediately check the notification and take the necessary action.

[1553] Step 2: Implementing countermeasures

[1554] User:

[1555] Based on the countermeasures presented by the server, appropriate care is provided for the animal's health condition and behavioral patterns.

[1556] For example, if an abnormality is detected, a veterinarian will be contacted and a detailed health check will be requested.

[1557] Step 3: Data feedback

[1558] User:

[1559] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[1560] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities in the future.

[1561] In this way, the entire process from collecting animal ecological data to analysis, notification, and response is carried out reliably.

[1562] Example 1

[1563] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1564] Modern animal husbandry environments require effective systems for real-time monitoring of animal health and behavioral patterns and early detection of abnormalities. Conventional methods have fragmented processes for data collection, analysis, and notification, making it difficult to respond immediately. They also lack an interface that allows users to intuitively grasp the health status of animals. To solve these problems and improve animal welfare, an integrated system must be built.

[1565] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1566] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for sending the generated alert to a terminal and the terminal notifying the user of it in real time, means for visually displaying the analysis results through a user interface and visualizing them as graphs or charts, and means for the user to refer to past data and analyze trends. This makes it possible to efficiently monitor the animal's health condition and behavioral patterns, detect abnormalities early, and take appropriate measures promptly.

[1567] A "sensing device" is a device that is attached to an animal and collects biological data such as heart rate, body temperature, and location information in real time.

[1568] "Video data" refers to digital data that records the movements and behavior of animals, captured using a video device such as a camera.

[1569] "Generative AI" is an artificial intelligence technology that analyzes ecological and video data to detect changes in animals' health and behavioral patterns.

[1570] A "database" is an information storage system for saving analysis results, and provides long-term data accumulation and backup functions.

[1571] An "alert" is a warning message that is generated when an abnormality is detected, and is a means of notifying the user.

[1572] A "terminal" is an interface device that allows users to use the system and receives and displays alerts and analysis results from the server.

[1573] A "user interface" is a collection of screens and functions that contain visual and operational elements that allow a user to interact with a system.

[1574] A "graph" is a diagram for visually displaying data, visualizing fluctuations along a time axis.

[1575] A "chart" is a graphic representation of numerical data visually, and includes formats such as bar graphs and line graphs.

[1576] A "trend" refers to a tendency or tendency observed from past data.

[1577] System Overview

[1578] This invention is a system for monitoring animal health and behavioral patterns in real time and providing appropriate care. It mainly consists of a sensing device, video analysis tools, generative AI, and a user interface. The purpose of this system is to improve animal welfare and optimize the housing environment.

[1579] Program Description

[1580] Server Processing

[1581] The server is the center of this system, and performs data collection, analysis, storage, and notification. Specifically, it performs the following processes:

[1582] 1. Data Collection

[1583] The server receives real-time biological data, such as heart rate, body temperature, and location information, from sensing devices attached to the animals. For example, it receives data that a lion's heart rate has increased from 60 bpm to 80 bpm.

[1584] The server receives video data from cameras installed in the enclosure, capturing footage of the lions' movements and postures every 10 seconds.

[1585] 2. Data Transmission

[1586] The server simultaneously receives the collected biological data and video data, so it needs to maintain stable data communication over the network. For example, the server synchronizes data with the device using Wi-Fi.

[1587] 3. Data Analysis

[1588] The generative AI model on the server analyzes the received biological data and video data in real time. Specifically, the AI ​​detects abnormal heart rate patterns and behavioral abnormalities based on the video data. For example, it detects stress by linking a sudden increase in heart rate with vigorous movements in the video data.

[1589] 4. Data storage and alert generation

[1590] The server stores the results of the analysis by the generative AI in a database. For example, the analysis results are stored in a database as a lion's health log, including heart rate, body temperature, and behavioral patterns.

[1591] The server generates an alert if it detects an abnormality. For example, if the heart rate is abnormally high, the server adds "Abnormal Heart Rate: Urgent" to the alert list.

[1592] 5. User Notices

[1593] The server sends the generated alerts to the device. For example, when a new alert is generated, the server notifies the user by push notification or email.

[1594] The device notifies the user of the received alert in real time. For example, the device may use the smartphone's push notification function to send the user a message saying, "The lion's heart rate is abnormal. Please check immediately."

[1595] Terminal handling

[1596] The terminal is an interface that allows users to use the system. Specifically, it includes the following processes:

[1597] 1. Receiving and displaying data

[1598] The device receives analysis results and alert information from the server, allowing users to understand the animal's condition in real time.

[1599] The data is displayed visually and visualized as graphs and charts, for example, showing fluctuations in heart rate or body temperature over time, allowing users to spot any abnormalities at a glance.

[1600] 2. Managing Notifications

[1601] The device immediately notifies the user of important alert information in a variety of ways, including push notifications and emails.

[1602] If an abnormality is detected, countermeasures are presented on the terminal, helping the user to take appropriate measures immediately.

[1603] User Action

[1604] Users use the system to manage the health and behavioral patterns of their animals, including the following processes:

[1605] 1. Data Monitoring

[1606] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[1607] For example, if a user is monitoring the health of a lion, they will receive a notification from the system if an abnormal heart rate or behavioral pattern is detected, allowing them to quickly schedule an examination or medical procedure.

[1608] 2. Data Management and Feedback

[1609] Users can view long-term data to identify trends in animal health and behavior, allowing them to provide preventative care.

[1610] If necessary, specific abnormal behaviors or health conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[1611] Specific examples

[1612] A specific example is a health management system for lions at a zoo. The server receives heart rate and body temperature data in real time from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, an alert is generated and notified to the user via their device. The user can immediately analyze the cause of the abnormality and take appropriate action. This system makes it possible to efficiently monitor the health of animals, detect abnormalities early, and respond quickly. This improves animal welfare and optimizes the captive environment.

[1613] Prompt Sentence Examples

[1614] "Analyze the heart rate and temperature data received from the sensing device attached to the lion, as well as the camera footage, to assess the lion's health and behavioral patterns. If you detect any abnormalities, state the reason and recommend a course of action."

[1615] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1616] Step 1:

[1617] Data collection

[1618] The server receives biological data such as heart rate, body temperature, and location information in real time from sensing devices attached to the animals. The input is data from various sensors, and the output is raw data stored in the server. Specifically, the server receives and temporarily stores data on heart rate increases from 60 bpm to 80 bpm and body temperature data every unit of time.

[1619] Step 2:

[1620] Video data collection

[1621] The server receives video data from cameras installed in the enclosure. The input is the video signal from the camera, and the output is the video data stored in the server. For example, the server captures and stores video of lions' movements every 10 seconds.

[1622] Step 3:

[1623] Data transmission

[1624] The server communicates via a network to stably receive the collected biological data and video data. The input is data from the sensing device and camera, and the output is the integrated data sent to the server. Specifically, the server synchronizes the real-time data using Wi-Fi.

[1625] Step 4:

[1626] Data analysis

[1627] The generative AI model on the server analyzes the received ecological data and video data in real time. The input is the ecological data and video data stored on the server, and the output is the analyzed health status and behavioral pattern changes. Specifically, the AI ​​analyzes sudden increases in heart rate and violent animal movements based on the video to detect stress or health abnormalities.

[1628] Step 5:

[1629] Data storage

[1630] The server stores the results of the analysis by the generative AI in a database. The input is the analyzed health data and behavioral pattern data, and the output is the historical data stored in the database. Specifically, the server adds logs of the lion's heart rate, body temperature, and behavioral patterns to the database.

[1631] Step 6:

[1632] Alert Generation

[1633] The server generates an alert if it detects an abnormality. The input is the abnormal data from the generation AI, and the output is a warning message added to the alert list. For example, if the heart rate is abnormally high, the server adds "Heart rate abnormal: emergency" to the alert list.

[1634] Step 7:

[1635] Alert Notification

[1636] The server sends the generated alert to the device. The input is the alert message, and the output is the alert notification sent to the device. Specifically, the server sends an emergency message to the user via push notification or email.

[1637] Step 8:

[1638] Real-time notifications

[1639] The terminal notifies the user of the received alert in real time. The input is the alert notification sent from the server, and the output is a warning message displayed on the user's terminal. For example, the terminal may display a message such as "The lion's heart rate is abnormal. Please check immediately."

[1640] Step 9:

[1641] Data Monitoring

[1642] The user uses a terminal to monitor the analysis results and alert information from the server in real time. The input is the analysis results displayed on the terminal's dashboard, and the output is the user's judgment and response. Specifically, the user looks at the graphs and charts on the terminal to identify abnormalities.

[1643] Step 10:

[1644] Data Management and Feedback

[1645] Users can view long-term data to understand trends in animal health and behavior. The input is historical database information, and the output is analysis results and feedback. Specifically, users can view a year's worth of health data in graphs and provide feedback to the system to improve the accuracy of the generative AI.

[1646] (Application example 1)

[1647] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1648] In recent years, as the complexity and operating hours of robots used in factories have increased, it has become increasingly important to monitor the status of robots in real time and detect abnormalities early. However, current systems lack a mechanism for immediately detecting and efficiently notifying robot abnormalities, which is a factor that affects production efficiency and safety. A solution to this issue is needed.

[1649] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1650] In this invention, the server includes means for collecting operational data of the object from the sensing device in real time, means for receiving video data and analyzing the work pattern of the object, means for analyzing the operational data and video data using a generation AI and detecting changes in the operating state and work pattern of the object, means for saving the analysis results in a database and generating an alert when an abnormality is detected, and means for notifying engineers of the analysis results and alert information. This makes it possible to monitor robot abnormalities in real time and to immediately notify engineers when an abnormality is detected.

[1651] A "sensing device" is a device that collects operational data and environmental data (temperature, vibration, power consumption, etc.) of an object in real time.

[1652] "Operation data" refers to data on the status of an object while it is in operation (such as temperature, vibration, and power consumption).

[1653] "Video data" refers to video information of an object being worked on, captured using photographic equipment such as a camera.

[1654] "Work pattern" refers to the pattern of actions and behaviors of an object when performing a specific task or operation.

[1655] "Generative AI" is artificial intelligence that analyzes changes in the operating status and work patterns of objects based on collected operational data and video data.

[1656] "Analysis results" refer to the information and insights obtained when operational data and video data are analyzed by the generating AI.

[1657] A "database" is a storage medium or system for long-term storage and management of analysis results and ecological data.

[1658] An "alert" is a warning notification that is generated when an abnormality is detected and is immediately reported to a technician.

[1659] An "engineer" is someone who receives analysis results and alert information and takes action.

[1660] A "dashboard" is a user interface that allows engineers to visually check analysis results and alert information.

[1661] System Overview

[1662] This invention is a system that enables preventive maintenance by monitoring the status and operation patterns of factory robots in real time and immediately notifying engineers if an abnormality is detected. The system mainly consists of a sensing device, a video analysis tool, a generative AI, and a user interface with a dashboard.

[1663] Program Description

[1664] Server Processing

[1665] The server is the center of the system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[1666] 1. Data Collection

[1667] The server receives real-time operational data such as temperature, vibration, and power consumption from sensing devices attached to factory robots, allowing the status of the target object to be grasped immediately.

[1668] The server receives video data from cameras installed in the factory and captures the robot's work patterns.

[1669] 2. Data Analysis

[1670] The collected operational and video data is analyzed by the generative AI, which uses this data to analyze the operating status and working patterns of the object and detect any abnormalities.

[1671] For example, if the AI ​​detects an abnormal rise in temperature, it determines whether this is a sign of overheating or abnormal operation.

[1672] 3. Data Retention and Notification

[1673] The analysis results are stored in a database that has long-term data accumulation and backup functions.

[1674] If an anomaly is detected, the server immediately generates an alert and notifies the technician. Alerts are categorized according to the type of anomaly and its urgency.

[1675] Terminal handling

[1676] The terminal is the interface through which the engineer uses the system. The process is explained below in natural language.

[1677] 1. Receiving and displaying data

[1678] The terminal receives analysis results and alert information from the server, allowing engineers to understand the status of the target object in real time.

[1679] The data is displayed visually and visualized on a dashboard as graphs and charts, for example showing temperature or vibration fluctuations over time, allowing engineers to spot anomalies at a glance.

[1680] 2. Managing Notifications

[1681] The device instantly notifies technicians of important alert information in a variety of ways, including push notifications and emails.

[1682] If an abnormality is detected, countermeasures are presented on the terminal, helping the technician to take appropriate measures immediately.

[1683] User Action

[1684] The user, as an engineer, uses the system to manage the operating status and work patterns of the factory robots. The process is explained below in natural language.

[1685] 1. Data Monitoring

[1686] Users can use their devices to monitor analysis results and alert information from the server in real time, enabling them to respond immediately when an abnormality is detected.

[1687] For example, if a user is monitoring the temperature status of a factory robot, if an abnormal temperature rise or vibration pattern is detected, the system will notify them and they can quickly take inspection or repair measures.

[1688] 2. Data Management and Feedback

[1689] Users can view long-term data to understand trends in the operating status and work patterns of their equipment, allowing them to provide preventative maintenance.

[1690] If necessary, specific abnormal behaviors or conditions can be fed back to the system, improving the analytical accuracy of the generative AI and making future anomaly detection more accurate.

[1691] Specific examples

[1692] A specific example is a robot maintenance system in a factory. The server receives temperature and vibration data in real time from sensing devices attached to the robot, and also captures camera footage. The generative AI analyzes this data and monitors the robot's operating status. If an abnormality is detected, an alert is generated and notified to the technician via the device. The technician can immediately analyze the cause of the abnormality and take appropriate action.

[1693] This system makes it possible to efficiently monitor the operating status of factory robots, detect abnormalities early, and respond quickly, thereby improving production efficiency and safety within the factory.

[1694] Prompt Sentence Examples

[1695] "The temperature of a factory robot is rising abnormally. An 85°C warning has been issued. Please provide instructions for generating an immediate alert and notifying a technician."

[1696] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1697] Step 1:

[1698] The server collects operational data such as temperature, vibration, and power consumption in real time from sensing devices attached to factory robots. This input data allows the robot's status to be constantly monitored. The collected data is initially processed on the server and converted into an appropriate format.

[1699] Step 2:

[1700] The server receives video data in real time from cameras installed in the factory. The video data is used to visually capture the robot's movements and work patterns. The received video is divided into frames and saved.

[1701] Step 3:

[1702] The server provides collected operational and video data as input to the generative AI model, which analyzes this data to detect the object's operating status and work patterns. If an abnormality is detected, the AI ​​model will recognize signs such as elevated temperatures or abnormal vibrations.

[1703] Step 4:

[1704] The server stores the analysis results of the generative AI model in a database. The stored data is organized with time information to facilitate future searches and analysis, while maintaining the integrity and security of the data.

[1705] Step 5:

[1706] If the generative AI model detects an anomaly, the server immediately generates an alert, generating an appropriate message depending on the type and urgency of the anomaly, which is then recorded in the alert log.

[1707] Step 6:

[1708] The server notifies the technician of the generated alert. The notification can be sent in the form of a push notification, email, SMS, etc. For example, in the case of a temperature abnormality, a message will be sent saying, "Robot A's temperature has reached 85°C. Immediate action is required."

[1709] Step 7:

[1710] The terminal visually displays the analysis results and alert information received from the server. The results are converted into graphs and charts on the user interface, allowing engineers to check them in real time. For example, fluctuations in temperature or vibration can be seen at a glance.

[1711] Step 8:

[1712] Users use their terminals to monitor and check the analysis results and alert information from the server. When an abnormality is detected, users can provide feedback to the system to take immediate action. Specifically, users can record actions such as "Check that Robot A's temperature is abnormal and check the cooling system."

[1713] Step 9:

[1714] Users can refer to long-term data to understand trends in the operating status and work patterns of their equipment, enabling them to plan preventative maintenance and improvements, thereby improving production efficiency and safety within their factories.

[1715] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1716] System Overview

[1717] This invention is a system for monitoring the health status and behavioral patterns of animals in real time and providing appropriate care. In particular, by combining it with an emotion engine, it is possible to recognize the user's emotional state and dynamically change the notification method and content of alert information based on that. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine, and aims to improve animal welfare and optimize the housing environment.

[1718] Program Description

[1719] Server Processing

[1720] The server is the central hub of the entire system, collecting, analyzing, storing, and notifying data. Below, we will explain this process in natural language.

[1721] 1. Data Collection

[1722] The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals.

[1723] The server receives video data from cameras installed in the breeding area.

[1724] Both data are stored in a temporary storage area.

[1725] 2. Data Preprocessing

[1726] The server cleans the collected biometric data and corrects and deletes inaccurate or missing data.

[1727] Noise is also removed from the video data, and frame correction is performed.

[1728] 3. Data Analysis

[1729] The server uses generative AI to analyze the biological and video data, detecting abnormalities in heart rate and body temperature and identifying behavioral patterns.

[1730] For example, if a lion's heart rate suddenly increases, it may be interpreted as a sign of stress or illness.

[1731] 4. Emotional Data Collection and Analysis

[1732] The server uses an emotion engine to collect the user's emotion data and analyze the user's emotional state.

[1733] For example, if the user is feeling stressed, the method of notifying the alert information is changed.

[1734] 5. Data Retention and Notification

[1735] The analysis results are stored in a database, allowing data to be accumulated over a long period of time.

[1736] If an abnormality is detected, the server generates an alert and sends it using a notification method that takes into account the user's emotional state.

[1737] Terminal handling

[1738] The terminal is the interface through which the user uses the system. The process is explained below in natural language.

[1739] 1. Receiving and displaying data

[1740] The terminal receives analysis results and alert information from the server in real time.

[1741] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[1742] 2. Displaying the user's emotional state

[1743] The device displays the user's emotional state and visualizes it along with the analysis results. By checking their own emotional state, users can make calm decisions.

[1744] 3. Managing Notifications

[1745] The notification method and content of alert information are dynamically changed depending on the user's emotional state. For example, if the user is feeling stressed, the notification may be changed to email, rather than prompting an immediate response.

[1746] User Action

[1747] Users can use this system to manage the health and behavioral patterns of their animals. The process is explained in natural language below.

[1748] 1. Data Monitoring

[1749] Users can monitor analysis results and alert information in real time via their terminals.

[1750] The user calmly judges whether there is anything abnormal while checking their own emotional state.

[1751] 2. Implementing emotional coping strategies

[1752] When implementing the displayed countermeasures, the user can take measures that take into account his or her own emotional state.

[1753] For example, it helps a user in a stressed state to take calm action.

[1754] 3. Data Feedback

[1755] Users can provide detailed feedback on abnormal behavior and health conditions to improve the analytical accuracy of the generative AI.

[1756] For example, by recording the progress after treatment and using this information as a reference for the next abnormality detection, response times can be shortened.

[1757] Specific examples

[1758] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. The generative AI analyzes this data and monitors the lions' health. If an abnormality is detected, the emotion engine recognizes the user's emotional state and changes the notification method for alert information. For example, if the user is feeling stressed, an email notification can be sent to avoid immediate response. This system makes it possible to efficiently monitor the health of animals while providing care that takes the user's emotional state into consideration.

[1759] The processing flow will be explained below.

[1760] Server Processing

[1761] Step 1: Collecting data from sensing devices

[1762] server:

[1763] The server receives real-time biological data such as heart rate, body temperature, and location information from sensing devices attached to the animals.

[1764] The collected data is stored in a temporary storage area.

[1765] Step 2: Collecting video data from the camera

[1766] server:

[1767] The server receives video data in real time from cameras installed within the breeding area.

[1768] Video data is also stored in the temporary storage area.

[1769] Step 3: Preprocessing the data

[1770] server:

[1771] The server cleans the biometric data, correcting and deleting inaccurate or missing data.

[1772] Noise removal and frame correction are also performed on the video data.

[1773] Step 4: Analysis of ecological and video data

[1774] server:

[1775] The collected and pre-processed data is analyzed using generative AI to detect abnormalities in heart rate and body temperature.

[1776] Animal behavior patterns are extracted from video data and abnormal behavior is detected.

[1777] Step 5: Emotional Data Collection and Analysis

[1778] server:

[1779] The emotion engine collects emotion data from the user's device, including facial expression recognition and voice analysis.

[1780] The server analyzes the collected emotional data and determines the user's stress level and level of urgency.

[1781] Step 6: Data storage and alert generation

[1782] server:

[1783] The analysis results are stored in a database, and data is accumulated over a long period of time.

[1784] If an abnormality is detected, an alert is generated, and the alert information includes the type of abnormality and its urgency.

[1785] Step 7: Sending notifications

[1786] server:

[1787] A notification method for alert information is dynamically selected based on the emotional state of a user.

[1788] For example, if the user is feeling stressed, the notification method can be changed to email, and immediate action will not be urged.

[1789] Terminal handling

[1790] Step 1: Receiving Data

[1791] Device:

[1792] The terminal receives analysis results and alert information from the server in real time.

[1793] Step 2: Visualize the data

[1794] Device:

[1795] Generate graphs and charts to visually display the received data, for example, showing fluctuations in heart rate or body temperature.

[1796] Displays a timeline of behavioral patterns and highlights abnormal behavior.

[1797] Step 3: Obtaining and displaying emotion data

[1798] Device:

[1799] The device analyzes and displays the user's emotional state in real time, using a camera and microphone to collect emotional data.

[1800] The user's current emotional state is displayed and visualized along with the analysis results.

[1801] Step 4: Manage notifications

[1802] Device:

[1803] The notification method and content of alert information are dynamically changed according to the user's emotional state.

[1804] For example, if the user is feeling stressed, the system will notify them in a quieter way, avoiding the audible alarm.

[1805] User Action

[1806] Step 1: Data monitoring

[1807] User:

[1808] Users can monitor analysis results and alert information in real time via their terminals.

[1809] Check your emotional state in conjunction with the alert information and calmly assess any abnormalities.

[1810] Step 2: Implementing countermeasures

[1811] User:

[1812] When implementing the displayed countermeasures, you can take into account your own emotional state.

[1813] For example, it helps a user in a stressed state to take calm action.

[1814] Step 3: Data feedback

[1815] User:

[1816] Detailed information about abnormal behavior and health conditions is fed back to the system, improving the analytical accuracy of the generative AI.

[1817] For example, the progress after treatment can be recorded and used as a reference for detecting abnormalities next time.

[1818] Based on this detailed processing flow, a series of processes from collecting animal biological data to analysis, notification, and response are reliably carried out while taking into account the emotional state of the user.

[1819] Example 2

[1820] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1821] Conventional animal health and behavioral pattern monitoring systems only detect abnormalities in animals and do not provide a notification method that takes into account the user's emotional state. As a result, if the user is feeling stressed, the notification may not be received properly, making it difficult to respond quickly. Furthermore, there is no way for the user to understand their own emotional state and deal with the situation calmly.

[1822] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1823] In this invention, the server includes means for collecting animal ecological data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecological data and video data using a generation AI to detect changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, means for analyzing the user's emotional state, and means for dynamically changing the alert notification method based on the user's emotional state. This enables efficient monitoring of the animal's health condition while enabling prompt and appropriate responses by using a notification method that takes the user's emotional state into consideration.

[1824] "Sensing device" refers to a device worn by an animal to collect biological data in real time.

[1825] "Biological Data" refers to information including physiological and behavioral data of an animal, such as the animal's heart rate, body temperature, and location.

[1826] "Video data" refers to video information of animal behavior and environment collected using a camera or other image capture device.

[1827] "Generative AI" refers to an artificial intelligence model for analyzing ecological and video data. This model has the ability to automatically analyze animal health and behavioral patterns.

[1828] A "database" refers to a data management system for storing analysis results and collected data, which allows for long-term data accumulation.

[1829] An "alert" is a notification that is generated when an abnormality is detected. This notification notifies the user that an abnormality has occurred and prompts them to take appropriate action.

[1830] "User emotional state" refers to data that indicates the psychological state and stress level of the user using the system.

[1831] "Emotion engine" refers to a system component that has the function of collecting and analyzing the user's emotional state.

[1832] A "dashboard" refers to a user interface that allows users to visually check analysis results and their own emotional state.

[1833] "Dynamic change of notification method" refers to changing the notification method and content of alert information in real time depending on the user's emotional state. Examples include email notifications and on-screen pop-up notifications.

[1834] Specific embodiments of the present invention are described below.

[1835] This system monitors the health and behavioral patterns of animals in real time and provides appropriate care. In particular, by combining it with an emotion engine, it can recognize the user's emotional state and dynamically change the appropriate notification method. This system consists of a sensing device, a video analysis tool, a generative AI, a user interface, and an emotion engine.

[1836] The server serves as the central hub of the entire system and performs the following processes: First, the server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals. These sensing devices can be ordinary heart rate monitors or temperature sensors. The server also streams video data from cameras installed in the animal enclosures and stores it in a temporary storage area. This video data is used to analyze the behavioral patterns of the animals.

[1837] The server then cleans the collected ecology data, correcting and deleting inaccurate or missing data. It also removes noise from the video data and performs frame correction to improve the accuracy of the analysis. Generative AI is then used to analyze the ecology data and video data. This generative AI detects abnormal values ​​in the animal's heart rate and body temperature and identifies behavioral patterns.

[1838] As a concrete example, consider the case of managing the health of lions at a zoo. The server receives real-time heart rate and body temperature data from sensing devices attached to the lions, and also captures camera footage. Generative AI analyzes this data and monitors the lions' health. For example, if a lion's heart rate suddenly rises, the server will interpret this as a sign of stress or illness.

[1839] The server also collects the user's emotional state through the user interface. The emotion engine evaluates the user's stress level based on the user's operation speed and response time. Based on this information, the server analyzes the user's emotional state and dynamically changes the notification method for alert information. For example, if the user is feeling stressed, the server can change the emergency alert notification to an email notification, avoiding the need for an immediate response.

[1840] The analysis results are stored in a database, allowing data to be accumulated over a long period of time. This allows for long-term monitoring of the animal's health. If an abnormality is detected, the server generates an alert and sends the alert information using a notification method that takes the user's emotional state into consideration.

[1841] The terminal is the interface through which users use the system, receiving analysis results and alert information from the server in real time. To visually display the received data, the terminal generates graphs and charts and provides information to the user. Furthermore, the terminal displays the user's emotional state, which is visualized together with the analysis results, helping the user to calmly determine whether an abnormality has occurred. The terminal also appropriately manages the notification method and content of alert information according to the user's emotional state.

[1842] Finally, users can use the system to manage their animals' health and behavioral patterns. They can monitor analysis results and alert information in real time, and calmly assess abnormalities while checking their own emotional state. Furthermore, users can provide detailed feedback on abnormal behavior and health conditions to improve the analysis accuracy of the generated AI.

[1843] Examples of prompts include:

[1844] "Lion's heart rate is significantly higher than normal. If this abnormality persists, immediate medical attention is required. Due to the user's high stress levels, a notification has been sent via email."

[1845] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1846] Step 1:

[1847] Data collection

[1848] Server operation: The server receives real-time heart rate, body temperature, and location information from sensing devices attached to the animals, as well as video data from cameras installed in the animal enclosures.

[1849] Input: Biological data from sensing devices and video data from cameras.

[1850] Output: Biological data and video data stored in temporary storage area.

[1851] Specific operation: The server collects data from the sensing device every 5 seconds and stores it in a temporary storage area. It also streams camera footage and stores it in the temporary storage area.

[1852] Step 2:

[1853] Data Preprocessing

[1854] Server operation: The server cleans the collected biological data, corrects and deletes inaccurate and missing data, and removes noise from video data and performs frame correction.

[1855] Input: Biological data and video data in temporary storage area.

[1856] Output: Preprocessed and clean data.

[1857] Specific operation: The server executes a data cleaning algorithm to remove inaccurate points in the biological data and applies a noise reduction filter to the video data.

[1858] Step 3:

[1859] Data analysis

[1860] Server operation: The server uses generative AI to analyze the ecology and video data, specifically detecting abnormalities in heart rate and body temperature and identifying animal behavior patterns.

[1861] Input: Preprocessed ecological data and video data.

[1862] Output: Anomaly detection report and specific information on behavioral patterns as analysis results.

[1863] Specific operation: The generative AI model is executed to generate alert data when the heart rate exceeds a certain range or when abnormal movement is detected from the video.

[1864] Step 4:

[1865] Emotion data collection and analysis

[1866] Server operation: The server uses an emotion engine to collect and analyze user emotion data. It monitors the user's operation speed and response time to evaluate their stress level.

[1867] Input: User operation data and response data.

[1868] Output: Parsed user emotional state data.

[1869] Specific operation: The emotion engine analyzes the log data of the user interface and quantifies the user's stress level.

[1870] Step 5:

[1871] Data Retention and Notification

[1872] Server operation: The server stores the analysis results in a database and generates an alert if an anomaly is detected. The alert dynamically changes the notification method taking into account the user's emotional state.

[1873] Input: Anomaly detection report, emotional state data.

[1874] Output: Data stored in the database and alert information sent.

[1875] Specific operation: The server saves the anomaly detection report in a database, and based on the analysis results of the emotion engine, selects an appropriate notification method, such as email or a pop-up notification on the screen, and sends an alert to the user.

[1876] Step 6:

[1877] User Interface Operations

[1878] Device operation: The device receives analysis results and alert information from the server in real time and visually displays them, as well as the user's emotional state.

[1879] Input: Analysis results and alert information from the server.

[1880] Output: Visualized analysis results, alert information, and user emotional state.

[1881] Specific operation: The device converts the received data into graphs and charts and displays them, showing the user's emotional state in real time.

[1882] Step 7:

[1883] Data Feedback

[1884] User Action: The user uses the system to monitor the health and behavioral patterns of the animals and provides detailed feedback on abnormal behavior and health status.

[1885] Input: Analysis results and alert information.

[1886] Output: Feedback data from users.

[1887] Specific operation: The user monitors the system and, if an abnormality is detected, takes countermeasures and inputs the results into the system as feedback.

[1888] (Application example 2)

[1889] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1890] Currently, there are systems that monitor the health and behavioral patterns of animals in real time, but they do not take into account the user's emotional state. This results in inappropriate notifications being sent when the user feels stressed, resulting in an inability to respond appropriately. Furthermore, there is a lack of technology that can effectively analyze biological data from sensors and camera footage to quickly detect abnormalities. Therefore, there is a need for flexible notifications that take into account the user's emotional state while effectively managing animal health.

[1891] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting animal ecology data from a sensing device in real time, means for receiving video data and analyzing the animal's behavioral patterns, means for analyzing the ecology data and video data using a generation AI and detecting changes in the animal's health condition and behavioral patterns, means for saving the analysis results in a database and generating an alert when an abnormality is detected, means for notifying the user of the analysis results and alert information, and means for dynamically changing the notification method based on the user's emotional state using an emotion engine that collects and analyzes the user's emotional data. This enables effective monitoring of the animal's health condition and flexible notification that takes the user's emotional state into consideration.

[1892] A "sensing device" is a device used to collect biological and operational data of animals and machines in real time.

[1893] "Video data" means visually recorded data of animal or machine behavior obtained by a camera or other visual sensor.

[1894] "Generative AI" is a technology that uses artificial intelligence techniques to analyze large amounts of data and detect abnormalities in health conditions and changes in behavioral patterns.

[1895] The "database" is a system for storing and managing analysis results, ecological data, and video data over the long term.

[1896] An "alert" is a warning message or signal that notifies the user when an abnormality is detected.

[1897] An "emotion engine" is a technology for analyzing a user's emotional state and assessing the user's emotions based on collected data.

[1898] A "prompt sentence" is a sentence used to explain to the user specific analysis results or abnormal behavior generated by the generative AI.

[1899] The "dashboard" is an interface that allows users to visually check, monitor, and manage analysis results and emotional states.

[1900] "Notification method" refers to the means by which alerts and information are communicated to users, such as email or instant messaging.

[1901] This invention provides a system that monitors the health status and behavior patterns of animals or machines in real time and provides effective countermeasures taking into account the emotional state of the user. The system includes a sensing device, a camera, a generative AI, an emotion engine, a database, a dashboard, and a notification means.

[1902] System Configuration

[1903] 1. Sensing Devices

[1904] This device is attached to animals or machines and collects biological and behavioral data such as heart rate, body temperature, and vibration in real time. The collected data is sent to a server.

[1905] 2. Camera

[1906] This device captures the target's movement patterns as video data, which is then sent to a server for analysis.

[1907] 3. Server

[1908] It is the central part of the entire system and performs the following processes:

[1909] Data collection from sensing devices and cameras.

[1910] Pre-processing of collected data (noise removal, data correction).

[1911] Data analysis using generative AI.

[1912] Generate alerts when anomalies are detected.

[1913] Analyzing user emotional states using an emotion engine.

[1914] Analysis results and alert information are stored in a database and sent to users.

[1915] Program processing explanation

[1916] 1. Data Collection

[1917] The server receives real-time data such as heart rate, body temperature, and vibration from sensing devices attached to animals and machines. It also receives video data from cameras installed in the breeding area and factory. This data is stored in a temporary storage area.

[1918] 2. Data Preprocessing

[1919] The server cleanses the collected data, correcting and deleting inaccurate or missing data, and also removes noise from video data and performs frame correction.

[1920] 3. Data Analysis

[1921] The server uses generative AI to analyze the biological data and video data. For example, if the temperature sensor data suddenly rises, it may indicate that the machine is overheating. This allows the server to detect changes in the health and behavior patterns of animals and machines.

[1922] 4. Emotional Data Collection and Analysis

[1923] The emotion engine collects user emotional data and analyzes their emotional state, allowing the system to take appropriate measures such as changing notification methods if the user is feeling stressed.

[1924] 5. Data Retention and Notification

[1925] The server stores the analysis results in a database and generates an alert if an abnormality is detected. The emotion engine takes the user's emotional state into account and dynamically changes the notification method and content. For example, if the user is feeling stressed, they can select to be notified by email.

[1926] Specific examples

[1927] In a factory application, the operating status of a robot is monitored. Data from vibration sensors and video data from cameras are sent to a server from sensing devices. A generative AI model analyzes this data and detects anomalies. An emotion engine evaluates the user's emotional state and selects an email notification if, for example, the user is experiencing high stress.

[1928] An example prompt is:

[1929] Sensor data: Temperature is over 80 degrees.

[1930] Camera footage analysis: Robot movement is unstable.

[1931] Emotion Engine Input: The user is concerned about the high temperature data from the sensor and the results of the video analysis.

[1932] This system allows for efficient health management of animals or machines, while also providing flexible responses that take into account the emotional state of the user.

[1933] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1934] Step 1: Data collection

[1935] The server receives biological data such as heart rate, body temperature, and vibration in real time from sensing devices attached to animals and machines. It also receives video data from cameras of the breeding areas and factory interiors. The input is numerical data of heart rate and body temperature and video data, and the output is raw data stored in a temporary storage area. Specifically, it retrieves data from sensors and cameras using HTTP requests.

[1936] Step 2: Preprocessing the data

[1937] The server cleans the collected biological data and video data, correcting and deleting inaccurate and missing data. It also performs noise removal and frame correction on video data. The input is raw data stored in a temporary storage area, and the output is cleaned, consistent data. Specific operations include applying data cleaning algorithms and noise removal filters.

[1938] Step 3: Data analysis

[1939] The server uses the generative AI model to analyze the cleaned biological data and video data. This detects abnormal values ​​in heart rate and body temperature and identifies changes in behavior patterns. The input is the pre-processed data, and the output is the analysis results. Specifically, the data is input into the generative AI model and the analysis results are obtained.

[1940] Step 4: Collect and analyze emotion data

[1941] The server uses an emotion engine to collect and analyze the user's emotional data. It evaluates the user's emotional state based on the text data and physical condition data entered into the system. The input is the user's text data and physical condition data, and the output is the analysis result of the user's emotional state. Specifically, the text and physical condition data are entered into the emotion engine, and an emotion evaluation result is obtained.

[1942] Step 5: Alerting and Notification

[1943] The server generates an alert if an anomaly is detected based on the results of data analysis and sentiment analysis. Furthermore, it uses an emotion engine to select a notification method that takes into account the user's emotional state, and notifies the user of the alert information. The input is the results of data analysis and sentiment analysis, and the output is the alert information sent to the user. Specifically, the server uses an anomaly detection algorithm to generate an alert and sends the notification via the selected method (email or instant message).

[1944] Step 6: Data storage and feedback

[1945] The server stores the analysis results and alert information in a database. Users can check the displayed information and provide feedback to help improve the accuracy of the generative AI model and emotion engine. The input is the analysis results and user feedback, and the output is an updated database. Specific operations include writing the analysis results and integrating the feedback data.

[1946] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1947] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1948] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1949] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1950] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1951] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1952] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1953] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, motorcycles, and other devices, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1954] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1955] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1956] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1957] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1958] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1959] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1960] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1961] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1962] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1963] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1964] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1965] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1966] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1967] The following is further disclosed regarding the above embodiment.

[1968] (Claim 1)

[1969] a means for collecting animal ecology data in real time from a sensing device;

[1970] means for receiving the video data and analyzing the animal's behavioral patterns;

[1971] Generative AI will analyze ecological data and video data to detect changes in animal health and behavioral patterns.

[1972] A means to store the analysis results in a database and generate an alert when an anomaly is detected.

[1973] a means for notifying a user of the analysis results and alert information;

[1974] A system including:

[1975] (Claim 2)

[1976] 10. The system of claim 1, wherein the generating AI includes means for analyzing animal behavior patterns in conjunction with video analysis tools.

[1977] (Claim 3)

[1978] 2. The system according to claim 1, further comprising means for providing a dashboard that enables a user to visually check the analysis results.

[1979] "Example 1"

[1980] (Claim 1)

[1981] a means for collecting animal ecology data in real time from a sensing device;

[1982] means for receiving the video data and analyzing the animal's behavioral patterns;

[1983] Generative AI will analyze ecological data and video data to detect changes in animal health and behavioral patterns.

[1984] A means to store the analysis results in a database and generate an alert when an anomaly is detected.

[1985] a means for notifying a user of the analysis results and alert information;

[1986] a means for transmitting the generated alert to a terminal and for the terminal to notify the user of the alert in real time;

[1987] A means to...

Claims

1. a means for collecting animal ecology data in real time from a sensing device; means for receiving the video data and analyzing the animal's behavioral patterns; Generative AI will analyze ecological data and video data to detect changes in animal health and behavioral patterns. A means to store the analysis results in a database and generate an alert when an anomaly is detected. a means for notifying a user of the analysis results and alert information; A system including:

2. 10. The system of claim 1, wherein the generating AI includes means for analyzing animal behavior patterns in conjunction with video analysis tools.

3. 2. The system according to claim 1, further comprising means for providing a dashboard that enables a user to visually confirm the analysis results.

Images