system

Abstract

We provide the system. [Solution] This is a video acquisition device for filming animal behavior, and includes means for installation within the animal's habitat. A processing device for analyzing captured video data in real time and analyzing animal behavior patterns, A notification device means that immediately notifies of an abnormality detected by the aforementioned analysis device, A recording device means for recording anomaly detection data and managing its history, A feedback device means for adjusting the AI ​​model based on feedback in order to improve the accuracy of anomaly detection, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the breeding management of animals, it is difficult to continuously monitor the health status and abnormal behaviors of all animals manually, and as a result, abnormalities may be overlooked or discovered late. Furthermore, if an animal's illness or injury is overlooked, the symptoms will worsen, making treatment difficult, and there is a risk of affecting other animals and causing large losses. As the above problems, an effective monitoring system for early detection of abnormalities in the health status of animals and reduction of the burden on breeders is required.

Means for Solving the Problems

[0005] To address these challenges, the system utilizes video acquisition equipment installed within the animals' habitat to capture their behavior, and a processing unit that analyzes the captured video data in real time. This processing unit analyzes the animals' behavioral patterns and provides a notification system that immediately alerts the user when an anomaly is detected. Furthermore, by incorporating a recording device that records anomaly detection data and manages its history, it is possible to understand long-term health trends of the animals based on past data. The system also includes a feedback device that adjusts the AI ​​model based on feedback to improve detection accuracy. In this way, the system reduces the burden on animal caretakers and prevents animal health problems at an early stage by continuously and efficiently monitoring the animals' health.

[0006] A "video acquisition device" is a device used to film animal behavior and acquire that video data in real time.

[0007] A "processing device" is a device that analyzes video data obtained from a video acquisition device and has the function of detecting and analyzing animal behavior patterns.

[0008] A "notification device" is a device that has the function of immediately informing the user of any abnormalities detected by the processing unit.

[0009] A "recording device" is a device that records detected anomalies and analysis results and saves them as historical data.

[0010] A "feedback device" is a device equipped with the function to improve the accuracy of anomaly detection by receiving feedback from users and adjusting the AI ​​model accordingly. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] To implement this invention, video acquisition devices are installed within the animal's habitat, and a program that processes the video data obtained from these devices is run on a server. This system is designed to monitor the health status and behavioral abnormalities of animals and features real-time data processing and notification capabilities.

[0033] First, the terminal (video acquisition device) captures the animal's everyday behavior. This data is transmitted to a server in real time, where the behavioral patterns are analyzed. In particular, by capturing changes in the animal's movement speed, trajectory, and posture, abnormalities that deviate from normal behavioral patterns are detected.

[0034] If the server detects an anomaly, it immediately notifies the user terminal of the anomaly information. This allows the user to identify the anomaly early and take prompt action. In addition, the detected anomaly data and analysis results are stored in the server's recording device and can later be used as a database for health management.

[0035] Furthermore, in this invention, a feedback device plays a crucial role in continuously improving the AI ​​model. Based on feedback data obtained from the user, the model can be adjusted to improve the accuracy of anomaly detection. Through this process, the system can more accurately monitor the health status of animals and support the decisions of animal caretakers.

[0036] As a concrete example, consider a system for monitoring the walking patterns of cattle on a farm. AI cameras installed in the cattle grazing area capture the cattle's movements. A server analyzes this data and, if it detects any abnormalities such as limping, immediately sends a notification to the farm owner's terminal. This enables early diagnosis and treatment, improving farm efficiency while maintaining the health of the cattle.

[0037] By utilizing this system, it becomes possible to detect health abnormalities in animals early, reduce the burden on animal caretakers, and maintain the animals' health at an optimal level.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The terminal (video acquisition device) is installed in the animal's living environment and continuously films the animal's behavior. The video data is transmitted to the server either in its original state or compressed.

[0041] Step 2:

[0042] The server receives the video data in real time and first performs preprocessing. Specifically, it extracts the characteristics of the animal's posture and movement frame by frame. Noise reduction and extraction of important feature points are performed during this process.

[0043] Step 3:

[0044] The server's AI model analyzes pre-processed feature data and compares it to the animal's normal behavior patterns. If an abnormal behavior pattern is detected, it is flagged as an anomaly, and a detailed evaluation is performed based on this flag.

[0045] Step 4:

[0046] The server immediately notifies the user terminal when an anomaly is detected. The notification includes information about the type of anomaly, the time it occurred, and its urgency.

[0047] Step 5:

[0048] Users receive notifications and, if necessary, contact a veterinarian or investigate the abnormal area. They can also consider and implement appropriate countermeasures.

[0049] Step 6:

[0050] The server records all anomaly detection data and response actions in a database. This historical data is used to analyze long-term trends in animal health and improve management strategies.

[0051] Step 7:

[0052] Users provide feedback to improve detection accuracy, and the server's AI model is updated based on this feedback. Through improvements to the AI ​​model, the anomaly detection capability is further enhanced, and the system becomes able to adapt to new situations.

[0053] (Example 1)

[0054] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0055] In modern animal husbandry and wildlife monitoring, the rapid and accurate detection of animal health conditions and behavioral abnormalities is crucial for reducing the burden on caretakers and ensuring animal safety. However, conventional systems lack real-time capabilities and accuracy, resulting in frequent false positives and detection delays. Furthermore, few systems currently incorporate automated model improvement using feedback.

[0056] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0057] In this invention, the server includes information processing means for analyzing captured image data in real time and analyzing the behavior patterns of organisms, communication means for immediately notifying of anomalies detected by the information processing means, and correction means for adjusting the learning model based on feedback to improve the accuracy of anomaly detection. This enables accurate anomaly detection in real time and continuous accuracy improvement using feedback.

[0058] "Image acquisition means" refers to a photographic device installed to record the movements of animals or living organisms, and is capable of continuously collecting data in the environment.

[0059] An "information processing device" is a processing device that analyzes acquired image data in real time and evaluates the movements and behaviors of animals, and has the function of analyzing data using AI models and algorithms.

[0060] "Communication means" refers to devices or systems that immediately notify users of abnormal information detected as a result of analysis, and that have the ability to transmit data using a network.

[0061] A "recording device" is a device used to store detected abnormal data and analysis results for later reference and analysis, and functions as a database.

[0062] A "correction mechanism" refers to a system or device that adjusts the AI ​​model based on feedback data to improve the accuracy of anomaly detection, and has a mechanism to improve the model's performance through automatic learning.

[0063] A "behavioral pattern" refers to a series of normal or abnormal behavioral patterns exhibited by animals or organisms, which can be identified through analysis.

[0064] A "learning model" is an artificial intelligence model used to analyze animal behavior and detect anomalies, and it is continuously adjusted to improve its accuracy based on data.

[0065] This invention is a system for monitoring the health status and behavioral abnormalities of animals in real time. The system consists of the following hardware and software.

[0066] First, the image acquisition device installed on the terminal records the animal's movements. This device consists of a camera capable of reliable recording even in outdoor environments. The camera can continuously capture the animal's posture, movement speed, and trajectory for 24 hours.

[0067] The acquired video data is transmitted to the server in real time. The server processes the acquired data using analysis software. Specifically, it uses the image analysis library "OpenCV" and the artificial intelligence framework "TENSORFLOW®" to analyze the animal's behavior patterns. This makes it possible to determine whether the behavior deviates from normal patterns.

[0068] If an anomaly is detected, the server will immediately notify the user terminal. The user terminal consists of application software running on a smartphone or computer, and is designed to allow for quick verification and response upon receiving the notification.

[0069] Furthermore, it includes a correction mechanism that adjusts the AI ​​model based on feedback, allowing for continuous improvement of the model's accuracy. Users can contribute to system improvement by providing feedback on the analysis results and anomaly notifications provided.

[0070] As a concrete example, in a system that monitors the walking patterns of cows on a farm, video recorded by a terminal is sent to a server, and if an abnormality such as limping is detected, a notification is sent to the farm owner's smartphone. In this way, it becomes possible to detect illnesses early and provide appropriate treatment.

[0071] Another example of inputting prompts into a generative AI model is a text-based prompt such as, "List specific abnormalities in the animal's behavioral patterns." This prompt allows the AI ​​model to provide information on specific abnormal behaviors, thereby supporting the user's decision-making.

[0072] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0073] Step 1:

[0074] The terminal captures animal behavior in real time. The camera device monitors the habitat of the designated animal and continuously acquires image data. In this step, the input is video from the camera, and the output is video data. Specifically, the movement of the animal is captured through the camera lens and recorded as video.

[0075] Step 2:

[0076] The terminal transmits the acquired video data to the server in real time. In this process, the video data obtained in step 1 is used as input and transferred as a digital signal. As a result of this operation, the video data that reaches the server is obtained as output. Specifically, data transfer is performed using a network protocol.

[0077] Step 3:

[0078] The server analyzes the received video data using a generating AI model. The input is the video data received from step 2. The AI ​​model evaluates the animal's movement speed, trajectory, and posture changes, and performs data processing to identify anomalies from normal behavior patterns. The output is the analysis results if an anomaly is detected. The AI ​​model performs frame-by-frame analysis using "TensorFlow" and "OpenCV".

[0079] Step 4:

[0080] If the server detects an anomaly based on the analysis results, it immediately notifies the user terminal. The input uses the anomaly detection results obtained in step 3, and data conversion is performed to inform the user via the notification system. The output is an anomaly notification that is sent to the user terminal. Specifically, the anomaly is communicated via email or application notification.

[0081] Step 5:

[0082] The user reviews the system's analysis results through the received anomaly notification. The input for this process is the notification information from step 4, and the user reviews the current situation based on the displayed information and considers appropriate actions. The output is the user's judgment and the execution of countermeasures. Specifically, the user uses a smartphone or PC to review the details and decide on necessary actions such as physical inspection or treatment.

[0083] Step 6:

[0084] The server collects data to improve the accuracy of the AI ​​model through feedback. The input is user-provided feedback, which is used to adjust the AI ​​model. The output is an improved anomaly detection model. Specifically, model updates and tests are performed based on the feedback. This process enables the system to perform more accurate anomaly detection.

[0085] (Application Example 1)

[0086] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] Modern factories require the efficient and safe operation of industrial moving parts (such as robotic arms). However, malfunctions or abnormalities in these moving parts can lead to decreased productivity and safety threats, necessitating a rapid response. Conventional systems often rely on manual detection of anomalies and proposal of countermeasures, resulting in limitations in response speed and accuracy.

[0088] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0089] In this invention, the server is a video acquisition device for capturing the behavior of a moving object, and includes means installed at the work location of the moving object, processing means for analyzing the captured video data in real time and analyzing the behavior pattern of the moving object, and notification means for immediately notifying of any abnormalities detected by the analysis device. This enables rapid and accurate detection of abnormalities in the moving object and allows for the proposal of appropriate countermeasures, thereby improving productivity and ensuring safety.

[0090] "Operating device" refers to any mechanical device or system that performs a specific action, including industrial robot arms and automated machinery.

[0091] "Image acquisition equipment" is a general term for devices that include cameras and sensors for capturing the behavior of moving objects and collecting the resulting video data.

[0092] A "workplace" refers to a specific area or zone where an object performs its actions, such as a factory production line or a dedicated work area.

[0093] A "processing device" is a computer system that analyzes video data obtained from a video acquisition device and classifies and evaluates the behavior patterns of moving objects.

[0094] A "notification device" is a digital device or communication system that immediately transmits abnormalities detected by a processing unit to the user.

[0095] A "recording device" is a storage system that manages anomaly detection data and its history, and stores the data to enable later analysis and reference.

[0096] A "feedback device" is a platform that collects user feedback information and adjusts machine learning models with the aim of improving the detection accuracy of the system.

[0097] A "machine learning model" is a general term for algorithms and digital models used to learn from data and perform pattern recognition and anomaly detection.

[0098] To implement this invention, a system including the following components is required. First, a "video acquisition device" is installed at the work location of the moving object. The video acquisition device captures the behavior of the moving object in real time and transmits the video data to a server for processing. The server uses image analysis libraries such as Python and OpenCV to analyze the captured video data. In particular, the analysis device extracts parameters such as the movement speed, trajectory, and posture of the moving object and uses a machine learning model to identify normal behavior patterns and abnormal patterns.

[0099] If an anomaly is detected as a result of the analysis, the server quickly sends an alert to the user's terminal via a "notification device." The notification device is compatible with industrial monitoring systems and mobile devices, enabling immediate feedback. Furthermore, the anomaly detection data is stored in a "recording device" and used later for historical management. This data can be used as feedback to improve the accuracy of the analysis, and the server adaptively improves the machine learning model using a "feedback device."

[0100] A concrete example is a scenario where a robotic arm positioned on a factory production line performs packaging tasks. If the arm makes an unexpected movement and stops during normal operation, the abnormality is detected, and a warning is immediately sent to a terminal. This allows the operator to respond quickly and resume normal production activities.

[0101] Examples of prompt statements include the following:

[0102] "How can I monitor the behavior patterns of a robotic arm and issue an alert if it deviates from normal operation?"

[0103] The introduction of this system will significantly improve the production efficiency and safety of the factory.

[0104] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0105] Step 1:

[0106] The terminal captures the behavior of the working object from a video acquisition device placed at the work area where the object is performing its task. The captured video data is the input, and it is collected in video format. The terminal transmits this data to the server in real time.

[0107] Step 2:

[0108] The server uses Python and OpenCV to split the received video data into frames for processing. This split video data is used as input, and features such as the movement speed, trajectory, and orientation of the moving object are extracted from each frame. This results in a detailed dataset of the object's behavior.

[0109] Step 3:

[0110] The server uses a generative AI model to analyze the extracted features. The goal is to identify abnormal patterns by comparing them with pre-trained normal operating patterns. The input is the feature data from step 2, and the output is whether or not an anomaly was detected and the type of anomaly.

[0111] Step 4:

[0112] If an anomaly is detected, the server immediately sends an alert to the user's terminal via a notification device. Here, the presence or absence of an anomaly is the input, and a warning message to the user is output. For example, a specific message such as "An anomaly has been detected in the robot arm" might be displayed.

[0113] Step 5:

[0114] The server stores detected anomaly data in a recording device. This data serves as historical data for long-term analysis and feedback. The input is detailed data of the anomaly detection, and the output is the stored database entry.

[0115] Step 6:

[0116] Users can provide evaluations and additional information regarding detection results through a feedback device. This feedback is input for system improvement, and the server uses it to adjust the machine learning model and improve the accuracy of anomaly detection. As a result, a more powerful model is output.

[0117] Through the above process, it becomes possible to monitor the behavior of the moving object in detail and to quickly detect and notify of any abnormalities.

[0118] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0119] To implement this invention, a video acquisition device is installed within the animal's habitat, and the video data obtained from the device is processed on a server. A processing device that analyzes the animal's behavior patterns detects abnormalities based on its movement speed, trajectory, and posture, and quickly notifies the user's terminal of this information. This allows the user to respond early to changes in the animal's health condition.

[0120] Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, thereby adjusting notification methods and recommendations according to the user's emotional state. Specifically, the server uses the emotion engine to recognize the user's emotional state from their voice and text. Based on the specific emotion, it then adjusts the wording and urgency level of notifications to provide the user with the most appropriate information.

[0121] For example, users identified as being under significant stress will receive notifications in a softer tone and receive further support when suggesting specific solutions. The emotion engine continues to learn from user feedback, which is also used to fine-tune the AI ​​model to improve the accuracy of notifications.

[0122] As a concrete example of this invention, consider its implementation in a pet hotel. An AI camera captures the behavior of dogs left at the pet hotel, and if the server detects abnormal behavior, a user acting as a hotel staff member receives a notification on their smartphone. If the user is busy and stressed, the server recognizes their emotions through an emotion engine and provides detailed support along with further notifications.

[0123] This system not only quickly detects abnormalities in animals, but also reduces the user's mental burden and supports efficient health management.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] The terminal (video acquisition device) is placed in the animal's living environment and continuously acquires the animal's behavior. This video data is transmitted to a server via the network.

[0127] Step 2:

[0128] The server preprocesses the received video data and extracts the animal's movement characteristics. This includes frame-by-frame posture recognition and movement pattern analysis, enabling the detection of abnormal movements.

[0129] Step 3:

[0130] The AI ​​model on the server analyzes animal behavior based on pre-processed data, distinguishing between normal and abnormal behavior. During this process, a behavioral prediction algorithm is used to assess the animal's health status.

[0131] Step 4:

[0132] If abnormal behavior is detected, the server will immediately notify the user terminal of the details of the anomaly. The notification will include the specific type of anomaly, the time it occurred, and the recommended next steps.

[0133] Step 5:

[0134] The server uses an emotion engine to analyze the user's emotional state. It recognizes emotions from the user's voice and text input, adjusts notification content based on the results, and shares information appropriately.

[0135] Step 6:

[0136] Users receive notifications and check the situation. Based on the notifications, which are adjusted by the emotion engine, they can consider and implement ways to alleviate tension and specific countermeasures.

[0137] Step 7:

[0138] The server records detected anomalies and user feedback in a database. This information is used to improve long-term animal health management and enhance the accuracy of AI models.

[0139] Step 8:

[0140] Based on user feedback, the server's AI and emotion engine undergo regular adjustments to improve the quality of anomaly detection and user notifications.

[0141] (Example 2)

[0142] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0143] Monitoring animal behavior and detecting abnormalities early is crucial for animal health management. However, conventional monitoring systems have problems such as difficulty in analyzing animal behavioral characteristics in detail and in not being able to quickly notify users of abnormalities. Furthermore, the information provided does not take into account the emotional state of the user receiving the notification, making it difficult to take appropriate measures.

[0144] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0145] In this invention, the server is an imaging device for capturing animal behavior, and includes means for being installed in the area of ​​the organism; analysis device means for immediately analyzing the captured visual data and analyzing the behavioral characteristics of the organism; and emotion analysis device means for analyzing the user's emotional information and adjusting the notification method. This enables detailed analysis of the animal's behavioral characteristics and rapid detection of abnormalities, and further enables the provision of accurate notifications and support according to the user's emotional state.

[0146] An "imaging device" is a device that records the behavior of animals and is installed in areas where living organisms reside.

[0147] "Visual data" refers to video information acquired by an imaging device.

[0148] An "analysis device" is a device that instantly analyzes captured visual data to identify the behavioral characteristics of organisms.

[0149] A "notification device" is a device that quickly informs the user of abnormal information detected by an analysis device.

[0150] A "storage device" is a device used to store anomaly detection data and maintain records.

[0151] A "regulating device" is a device that adjusts a machine learning model based on feedback in order to improve the accuracy of anomaly detection.

[0152] An "emotion analysis device" is a device that analyzes the user's emotional information and adjusts the method of communication accordingly.

[0153] "Behavioral characteristics of an organism" refers to its usual behavioral patterns, including movement speed, route, and posture.

[0154] To implement this invention, it is necessary to install an imaging device in an area where animals live. This imaging device has the function of recording animal behavior and collecting visual data. The server acquires visual data from the imaging device in real time and processes the data using advanced image analysis algorithms. This allows for the analysis of animal behavioral characteristics and the detection of anomalies. For specific analysis, image processing libraries such as OpenCV can be used.

[0155] The server quickly sends detected anomaly information to the user's device. The device immediately notifies the user of the anomaly using push notification technology. Asynchronous communication is achieved by utilizing services such as Firebase Cloud Messaging. In addition, to take into account the user's stress and emotional state, the server uses an emotion analysis device to identify emotions from the user's voice and text. Machine learning tools such as TensorFlow can be used for emotion analysis.

[0156] Furthermore, based on the results of emotion analysis, customized notifications are provided according to the user's emotional state. This allows users to take appropriate action according to the animal's health condition. For example, for users experiencing significant stress, a notification message is generated that gently informs them of the situation and suggests specific countermeasures.

[0157] For example, if this system is used in a pet hotel, hotel staff can immediately identify any abnormal behavior in the animals being cared for and take necessary action. If a user is experiencing stress, the emotion analysis device will identify this and provide support with more helpful information.

[0158] By using the following example prompts, the emotion analyzer can perform highly accurate analysis. The notification wording can be adjusted based on scenarios such as "when the user is speaking" or "when the text message contains angry language." In this way, the present invention combines animal behavior monitoring with analysis of user emotional responses to provide an effective health management solution.

[0159] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0160] Step 1:

[0161] The server acquires visual data in real time from imaging devices installed within the animals' habitat. The input is a raw video stream from the imaging devices, which the server efficiently decodes and stores in memory. Specifically, it periodically captures frames of visual data and stores them in a high-speed cache. The output is video data in a format suitable for analysis.

[0162] Step 2:

[0163] The server processes the acquired video data using an analysis device. The input is the visual data obtained in step 1, and features such as the animal's movement speed, path, and posture are extracted through an image processing algorithm. Specifically, the OpenCV library is used to detect the outline of the animal in the video and calculate its motion vector. The output is numerical data indicating the animal's behavioral characteristics.

[0164] Step 3:

[0165] The server analyzes the behavioral characteristics data obtained in step 2 and detects anomalies. The input is numerical data obtained through feature extraction. A generative AI model is used to compare it with normal patterns and identify abnormal behavior. Specifically, an anomaly score is calculated to determine abnormal values, and a warning is generated if the threshold is exceeded. The output is a list of anomaly detection flags and anomaly details.

[0166] Step 4:

[0167] The server prepares to notify the terminal of the anomaly detection information. The input is the anomaly detection flag identified in step 3 and its details. A notification service (e.g., Firebase Cloud Messaging) is used to generate a push notification and send it to the user's terminal. Specifically, the notification message is formatted and the send request is processed asynchronously. The output is the notification message sent to the user's terminal.

[0168] Step 5:

[0169] The server uses an emotion analysis device to analyze the user's emotional state. Input consists of voice data and text messages provided by the user. Natural language processing (NLP) and machine learning models are used to calculate an emotion score. Specifically, speech recognition technology is used to convert speech to text, which is then analyzed by the emotion analysis algorithm. The output is the emotion evaluation score and its interpretation.

[0170] Step 6:

[0171] The server adjusts the notification content appropriately based on the user's sentiment rating score. The input is the sentiment score obtained in step 5. It generates prompts to change the wording and urgency of the notification, providing optimal information. Specifically, it selects a preset template and applies customizations according to the sentiment. The output is the adjusted notification message.

[0172] (Application Example 2)

[0173] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0174] The operation of the target objects, particularly industrial machinery and robots, requires rapid detection of abnormalities and appropriate countermeasures. However, current systems have challenges such as delayed anomaly detection and one-size-fits-all notifications to administrators, making it difficult to respond quickly and accurately in certain situations. Furthermore, methods for reducing the mental burden on administrators have not been sufficiently established.

[0175] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0176] In this invention, the server is a video acquisition device for capturing the movement of an object, and includes means installed within the movement range of the object, processing means for analyzing the captured video data in real time and analyzing the movement pattern of the object, and emotion analysis device means for analyzing the user's emotional state and adjusting the notification content. This enables rapid and accurate detection and notification of abnormalities, as well as the provision of flexible notification content according to the administrator's emotional state.

[0177] A "video acquisition device" is a device installed to film the movement of an object, and it records the movement within its range.

[0178] A "processing device" is a device that analyzes captured video data in real time and detects the movement patterns of an object.

[0179] A "notification device" is a device that immediately informs administrators of anomalies detected by an analysis device.

[0180] An "emotion analysis device" is a device that analyzes the user's emotional state and adjusts the content of notifications accordingly.

[0181] A "recording device" is a device used to record anomaly detection data and manage its history.

[0182] A "feedback device" is a device that has the function of adjusting the generated AI model based on feedback information in order to improve the accuracy of anomaly detection.

[0183] A "generative AI model" is an artificial intelligence model that can be adjusted to improve the accuracy of anomaly detection based on feedback.

[0184] To implement this invention, a camera is first required to identify the target objects, such as machinery and robots within a factory, and to monitor their operation in real time. This camera is installed as an AI camera and covers the operating range of the target objects. In addition, a server is required to process the captured video data and detect anomalies; this server acts as a processing unit.

[0185] The server uses OpenCV to analyze video data and analyzes behavioral patterns in real time. Based on these behavioral patterns, a generative AI model using TensorFlow detects anomalies and immediately notifies the administrator. Furthermore, the server uses an NLP library to analyze the administrator's emotional state and flexibly adjusts the notification content when the user is experiencing stress.

[0186] Furthermore, this system utilizes a recording device to record anomaly detection data, and a feedback device adjusts the generated AI model based on past anomaly detection records, thereby improving the accuracy of anomaly detection.

[0187] As a concrete example, the system detects abnormal vibrations occurring on a conveyor belt in a factory and uses this information to notify the manager of a potential overload condition. If the manager is under stress, the notification can be made in a gentler tone, such as, "There are some slight vibrations on the conveyor belt. No immediate action is required, but please check it periodically."

[0188] An example of a prompt message would be: "Use the AI ​​camera to monitor the robot's vibration patterns, and if an anomaly is detected, adjust the notification method according to the administrator's emotional state. Emotion analysis will help select the most appropriate notification format."

[0189] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0190] Step 1:

[0191] The AI ​​camera captures the movements of the target object, such as a machine or robot.

[0192] The input is real-time video of the object, and the output is recorded video data. The camera continuously acquires video and transmits the data to the server in real time.

[0193] Step 2:

[0194] The server uses OpenCV to analyze the video data and extract the motion patterns of the target object.

[0195] The input is the video data obtained in step 1, and the output is the motion pattern data. The server uses computer vision technology to extract motion features and convert the data into a format usable in the next processing step.

[0196] Step 3:

[0197] The server uses TensorFlow to perform anomaly detection using a generative AI model.

[0198] The input is the operation pattern data obtained in step 2, and the output is the anomaly detection result data. In this step, anomalies are identified by comparing them with known normal patterns. If an anomaly is detected, the information is sent to the next notification step.

[0199] Step 4:

[0200] The server uses an NLP library to analyze the administrator's emotional state.

[0201] The input is voice and text data from the administrator, and the output is emotional state data. Based on this emotional data, the server assesses the administrator's current stress level and prepares to generate appropriate notification content accordingly.

[0202] Step 5:

[0203] The server generates notification content tailored to the anomaly and the administrator's emotional state, and sends it to the administrator's terminal.

[0204] The input consists of the anomaly detection result data from step 3 and the emotional state data from step 4, and the output is a notification message to the administrator. This notification is flexibly adjusted according to the administrator's status and sent immediately.

[0205] Step 6:

[0206] The server saves the anomaly detection data to a recording device, and the AI ​​model is readjusted by a feedback device.

[0207] The input is anomaly detection result data, and the output is an updated generative AI model. In this step, the history is saved and used as feedback to train the AI ​​model, thereby improving the model's accuracy.

[0208] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0209] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0210] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0211] [Second Embodiment]

[0212] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0213] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0214] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0215] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0216] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0217] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0218] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0219] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0220] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0221] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0222] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0223] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0224] To implement this invention, video acquisition devices are installed within the animal's habitat, and a program that processes the video data obtained from these devices is run on a server. This system is designed to monitor the health status and behavioral abnormalities of animals and features real-time data processing and notification capabilities.

[0225] First, the terminal (video acquisition device) captures the animal's everyday behavior. This data is transmitted to a server in real time, where the behavioral patterns are analyzed. In particular, by capturing changes in the animal's movement speed, trajectory, and posture, abnormalities that deviate from normal behavioral patterns are detected.

[0226] If the server detects an anomaly, it immediately notifies the user terminal of the anomaly information. This allows the user to identify the anomaly early and take prompt action. In addition, the detected anomaly data and analysis results are stored in the server's recording device and can later be used as a database for health management.

[0227] Furthermore, in this invention, a feedback device plays a crucial role in continuously improving the AI ​​model. Based on feedback data obtained from the user, the model can be adjusted to improve the accuracy of anomaly detection. Through this process, the system can more accurately monitor the health status of animals and support the decisions of animal caretakers.

[0228] As a concrete example, consider a system for monitoring the walking patterns of cattle on a farm. AI cameras installed in the cattle grazing area capture the cattle's movements. A server analyzes this data and, if it detects any abnormalities such as limping, immediately sends a notification to the farm owner's terminal. This enables early diagnosis and treatment, improving farm efficiency while maintaining the health of the cattle.

[0229] By utilizing this system, it becomes possible to detect health abnormalities in animals early, reduce the burden on animal caretakers, and maintain the animals' health at an optimal level.

[0230] The following describes the processing flow.

[0231] Step 1:

[0232] The terminal (video acquisition device) is installed in the animal's living environment and continuously films the animal's behavior. The video data is transmitted to the server either in its original state or compressed.

[0233] Step 2:

[0234] The server receives the video data in real time and first performs preprocessing. Specifically, it extracts the characteristics of the animal's posture and movement frame by frame. Noise reduction and extraction of important feature points are performed during this process.

[0235] Step 3:

[0236] The server's AI model analyzes pre-processed feature data and compares it to the animal's normal behavior patterns. If an abnormal behavior pattern is detected, it is flagged as an anomaly, and a detailed evaluation is performed based on this flag.

[0237] Step 4:

[0238] The server immediately notifies the user terminal when an anomaly is detected. The notification includes information about the type of anomaly, the time it occurred, and its urgency.

[0239] Step 5:

[0240] Users receive notifications and, if necessary, contact a veterinarian or investigate the abnormal area. They can also consider and implement appropriate countermeasures.

[0241] Step 6:

[0242] The server records all anomaly detection data and response actions in a database. This historical data is used to analyze long-term trends in animal health and improve management strategies.

[0243] Step 7:

[0244] Users provide feedback to improve detection accuracy, and the server's AI model is updated based on this feedback. Through improvements to the AI ​​model, the anomaly detection capability is further enhanced, and the system becomes able to adapt to new situations.

[0245] (Example 1)

[0246] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0247] In modern animal husbandry and wildlife monitoring, the rapid and accurate detection of animal health conditions and behavioral abnormalities is crucial for reducing the burden on caretakers and ensuring animal safety. However, conventional systems lack real-time capabilities and accuracy, resulting in frequent false positives and detection delays. Furthermore, few systems currently incorporate automated model improvement using feedback.

[0248] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0249] In this invention, the server includes information processing means for analyzing captured image data in real time and analyzing the behavior patterns of organisms, communication means for immediately notifying of anomalies detected by the information processing means, and correction means for adjusting the learning model based on feedback to improve the accuracy of anomaly detection. This enables accurate anomaly detection in real time and continuous accuracy improvement using feedback.

[0250] "Image acquisition means" refers to a photographic device installed to record the movements of animals or living organisms, and is capable of continuously collecting data in the environment.

[0251] An "information processing device" is a processing device that analyzes acquired image data in real time and evaluates the movements and behaviors of animals, and has the function of analyzing data using AI models and algorithms.

[0252] "Communication means" refers to devices or systems that immediately notify users of abnormal information detected as a result of analysis, and that have the ability to transmit data using a network.

[0253] A "recording device" is a device used to store detected abnormal data and analysis results for later reference and analysis, and functions as a database.

[0254] A "correction mechanism" refers to a system or device that adjusts the AI ​​model based on feedback data to improve the accuracy of anomaly detection, and has a mechanism to improve the model's performance through automatic learning.

[0255] A "behavioral pattern" refers to a series of normal or abnormal behavioral patterns exhibited by animals or organisms, which can be identified through analysis.

[0256] A "learning model" is an artificial intelligence model used to analyze animal behavior and detect anomalies, and it is continuously adjusted to improve its accuracy based on data.

[0257] This invention is a system for monitoring the health status and behavioral abnormalities of animals in real time. The system consists of the following hardware and software.

[0258] First, the image acquisition device installed on the terminal records the animal's movements. This device consists of a camera capable of reliable recording even in outdoor environments. The camera can continuously capture the animal's posture, movement speed, and trajectory for 24 hours.

[0259] The acquired video data is transmitted to the server in real time. The server processes the acquired data using analysis software. Specifically, it uses the image analysis library "OpenCV" and the artificial intelligence framework "TensorFlow" to analyze the animal's behavior patterns. This makes it possible to determine whether the behavior deviates from normal patterns.

[0260] If an anomaly is detected, the server will immediately notify the user terminal. The user terminal consists of application software running on a smartphone or computer, and is designed to allow for quick verification and response upon receiving the notification.

[0261] Furthermore, it includes a correction mechanism that adjusts the AI ​​model based on feedback, allowing for continuous improvement of the model's accuracy. Users can contribute to system improvement by providing feedback on the analysis results and anomaly notifications provided.

[0262] As a concrete example, in a system that monitors the walking patterns of cows on a farm, video recorded by a terminal is sent to a server, and if an abnormality such as limping is detected, a notification is sent to the farm owner's smartphone. In this way, it becomes possible to detect illnesses early and provide appropriate treatment.

[0263] Another example of inputting prompts into a generative AI model is a text-based prompt such as, "List specific abnormalities in the animal's behavioral patterns." This prompt allows the AI ​​model to provide information on specific abnormal behaviors, thereby supporting the user's decision-making.

[0264] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0265] Step 1:

[0266] The terminal captures animal behavior in real time. The camera device monitors the habitat of the designated animal and continuously acquires image data. In this step, the input is video from the camera, and the output is video data. Specifically, the movement of the animal is captured through the camera lens and recorded as video.

[0267] Step 2:

[0268] The terminal transmits the acquired video data to the server in real time. In this process, the video data obtained in step 1 is used as input and transferred as a digital signal. As a result of this operation, the video data that reaches the server is obtained as output. Specifically, data transfer is performed using a network protocol.

[0269] Step 3:

[0270] The server analyzes the received video data using a generating AI model. The input is the video data received from step 2. The AI ​​model evaluates the animal's movement speed, trajectory, and posture changes, and performs data processing to identify anomalies from normal behavior patterns. The output is the analysis results if an anomaly is detected. The AI ​​model performs frame-by-frame analysis using "TensorFlow" and "OpenCV".

[0271] Step 4:

[0272] If the server detects an anomaly based on the analysis results, it immediately notifies the user terminal. The input uses the anomaly detection results obtained in step 3, and data conversion is performed to inform the user via the notification system. The output is an anomaly notification that is sent to the user terminal. Specifically, the anomaly is communicated via email or application notification.

[0273] Step 5:

[0274] The user reviews the system's analysis results through the received anomaly notification. The input for this process is the notification information from step 4, and the user reviews the current situation based on the displayed information and considers appropriate actions. The output is the user's judgment and the execution of countermeasures. Specifically, the user uses a smartphone or PC to review the details and decide on necessary actions such as physical inspection or treatment.

[0275] Step 6:

[0276] The server collects data to improve the accuracy of the AI ​​model through feedback. The input is user-provided feedback, which is used to adjust the AI ​​model. The output is an improved anomaly detection model. Specifically, model updates and tests are performed based on the feedback. This process enables the system to perform more accurate anomaly detection.

[0277] (Application Example 1)

[0278] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0279] Modern factories require the efficient and safe operation of industrial moving parts (such as robotic arms). However, malfunctions or abnormalities in these moving parts can lead to decreased productivity and safety threats, necessitating a rapid response. Conventional systems often rely on manual detection of anomalies and proposal of countermeasures, resulting in limitations in response speed and accuracy.

[0280] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0281] In this invention, the server is a video acquisition device for capturing the behavior of a moving object, and includes means installed at the work location of the moving object, processing means for analyzing the captured video data in real time and analyzing the behavior pattern of the moving object, and notification means for immediately notifying of any abnormalities detected by the analysis device. This enables rapid and accurate detection of abnormalities in the moving object and allows for the proposal of appropriate countermeasures, thereby improving productivity and ensuring safety.

[0282] "Operating device" refers to any mechanical device or system that performs a specific action, including industrial robot arms and automated machinery.

[0283] The "image acquisition device" is a general term for devices including cameras and sensors for shooting the behavior of a moving object and collecting its video data.

[0284] The "workplace" refers to a specific area or region where a moving object operates, such as a production line in a factory or a dedicated work area.

[0285] The "processing device" is a computer system for analyzing the video data obtained from the image acquisition device and classifying and evaluating the behavior pattern of the moving object.

[0286] The "notification device" is a digital device or communication system for immediately transmitting the abnormalities detected by the processing device to the user.

[0287] The "recording device" is a storage system for managing abnormality detection data and its history and storing data for later analysis and reference.

[0288] The "feedback device" is a platform for collecting feedback information from users for the purpose of improving the detection accuracy of the system and adjusting the machine learning model.

[0289] The "machine learning model" is a general term for algorithms and digital models used to learn from data and perform pattern recognition and anomaly detection.

[0290] To implement this invention, a system including the following components is required. First, an "image acquisition device" is installed at the workplace of the moving object. The image acquisition device plays a role of shooting the behavior of the moving object in real time and transmitting the video data to a server for processing. The server utilizes image analysis libraries such as Python and OpenCV to analyze the captured video data. In particular, the analysis device extracts parameters such as the moving speed, trajectory, and posture of the moving object, and uses a machine learning model to identify normal behavior patterns and abnormal patterns.

[0291] If an anomaly is detected as a result of the analysis, the server quickly sends an alert to the user's terminal via a "notification device." The notification device is compatible with industrial monitoring systems and mobile devices, enabling immediate feedback. Furthermore, the anomaly detection data is stored in a "recording device" and used later for historical management. This data can be used as feedback to improve the accuracy of the analysis, and the server adaptively improves the machine learning model using a "feedback device."

[0292] A concrete example is a scenario where a robotic arm positioned on a factory production line performs packaging tasks. If the arm makes an unexpected movement and stops during normal operation, the abnormality is detected, and a warning is immediately sent to a terminal. This allows the operator to respond quickly and resume normal production activities.

[0293] Examples of prompt statements include the following:

[0294] "How can I monitor the behavior patterns of a robotic arm and issue an alert if it deviates from normal operation?"

[0295] The introduction of this system will significantly improve the production efficiency and safety of the factory.

[0296] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0297] Step 1:

[0298] The terminal captures the behavior of the working object from a video acquisition device placed at the work area where the object is performing its task. The captured video data is the input, and it is collected in video format. The terminal transmits this data to the server in real time.

[0299] Step 2:

[0300] The server uses Python and OpenCV to split video data frame by frame in order to process the received video data. Using this split video data as the input for processing, it extracts feature quantities such as the moving speed, trajectory, and posture of the moving object from each frame. As a result, a detailed dataset of the behavior of the moving object is output.

[0301] Step 3:

[0302] The server utilizes a generative AI model to analyze the extracted feature quantities. At this time, the purpose is to compare with a pre-learned normal motion pattern and identify abnormal patterns. The input is the feature quantity data from Step 2, and the output is the presence or absence of anomaly detection and the type of anomaly.

[0303] Step 4:

[0304] When an anomaly is detected, the server immediately sends an alert to the user's terminal via the notification device. Here, the presence or absence of an anomaly is the input, and a warning message for the user is output. For example, a specific message such as "An anomaly has been detected in the robotic arm" is displayed.

[0305] Step 5:

[0306] The server stores the detected anomaly data in the recording device. This data becomes historical data for long-term analysis and feedback utilization. The input is the detailed data of anomaly detection, and the output is the saved database entry.

[0307] Step 6:

[0308] The user can provide an evaluation of the detection result and additional information through the feedback device. This is input as feedback information for system improvement, and the server adjusts the machine learning model based on this to improve the accuracy of anomaly detection. As a result, a model with improved performance is output.

[0309] Through the above process, it becomes possible to monitor the behavior of the moving object in detail and to quickly detect and notify of any abnormalities.

[0310] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0311] To implement this invention, a video acquisition device is installed within the animal's habitat, and the video data obtained from the device is processed on a server. A processing device that analyzes the animal's behavior patterns detects abnormalities based on its movement speed, trajectory, and posture, and quickly notifies the user's terminal of this information. This allows the user to respond early to changes in the animal's health condition.

[0312] Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, thereby adjusting notification methods and recommendations according to the user's emotional state. Specifically, the server uses the emotion engine to recognize the user's emotional state from their voice and text. Based on the specific emotion, it then adjusts the wording and urgency level of notifications to provide the user with the most appropriate information.

[0313] For example, users identified as being under significant stress will receive notifications in a softer tone and receive further support when suggesting specific solutions. The emotion engine continues to learn from user feedback, which is also used to fine-tune the AI ​​model to improve the accuracy of notifications.

[0314] As a concrete example of this invention, consider its implementation in a pet hotel. An AI camera captures the behavior of dogs left at the pet hotel, and if the server detects abnormal behavior, a user acting as a hotel staff member receives a notification on their smartphone. If the user is busy and stressed, the server recognizes their emotions through an emotion engine and provides detailed support along with further notifications.

[0315] This system not only quickly detects abnormalities in animals, but also reduces the user's mental burden and supports efficient health management.

[0316] The following describes the processing flow.

[0317] Step 1:

[0318] The terminal (video acquisition device) is placed in the animal's living environment and continuously acquires the animal's behavior. This video data is transmitted to a server via the network.

[0319] Step 2:

[0320] The server preprocesses the received video data and extracts the animal's movement characteristics. This includes frame-by-frame posture recognition and movement pattern analysis, enabling the detection of abnormal movements.

[0321] Step 3:

[0322] The AI ​​model on the server analyzes animal behavior based on pre-processed data, distinguishing between normal and abnormal behavior. During this process, a behavioral prediction algorithm is used to assess the animal's health status.

[0323] Step 4:

[0324] If abnormal behavior is detected, the server will immediately notify the user terminal of the details of the anomaly. The notification will include the specific type of anomaly, the time it occurred, and the recommended next steps.

[0325] Step 5:

[0326] The server uses an emotion engine to analyze the user's emotional state. It recognizes emotions from the user's voice and text input, adjusts notification content based on the results, and shares information appropriately.

[0327] Step 6:

[0328] Users receive notifications and check the situation. Based on the notifications, which are adjusted by the emotion engine, they can consider and implement ways to alleviate tension and specific countermeasures.

[0329] Step 7:

[0330] The server records detected anomalies and user feedback in a database. This information is used to improve long-term animal health management and enhance the accuracy of AI models.

[0331] Step 8:

[0332] Based on user feedback, the server's AI and emotion engine undergo regular adjustments to improve the quality of anomaly detection and user notifications.

[0333] (Example 2)

[0334] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0335] Monitoring animal behavior and detecting abnormalities early is crucial for animal health management. However, conventional monitoring systems have problems such as difficulty in analyzing animal behavioral characteristics in detail and in not being able to quickly notify users of abnormalities. Furthermore, the information provided does not take into account the emotional state of the user receiving the notification, making it difficult to take appropriate measures.

[0336] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0337] In this invention, the server is an imaging device for capturing animal behavior, and includes means for being installed in the area of ​​the organism; analysis device means for immediately analyzing the captured visual data and analyzing the behavioral characteristics of the organism; and emotion analysis device means for analyzing the user's emotional information and adjusting the notification method. This enables detailed analysis of the animal's behavioral characteristics and rapid detection of abnormalities, and further enables the provision of accurate notifications and support according to the user's emotional state.

[0338] An "imaging device" is a device that records the behavior of animals and is installed in areas where living organisms reside.

[0339] "Visual data" refers to video information acquired by an imaging device.

[0340] An "analysis device" is a device that instantly analyzes captured visual data to identify the behavioral characteristics of organisms.

[0341] A "notification device" is a device that quickly informs the user of abnormal information detected by an analysis device.

[0342] A "storage device" is a device used to store anomaly detection data and maintain records.

[0343] A "regulating device" is a device that adjusts a machine learning model based on feedback in order to improve the accuracy of anomaly detection.

[0344] An "emotion analysis device" is a device that analyzes the user's emotional information and adjusts the method of communication accordingly.

[0345] "Behavioral characteristics of an organism" refers to its usual behavioral patterns, including movement speed, route, and posture.

[0346] To implement this invention, it is necessary to install an imaging device in an area where animals live. This imaging device has the function of recording animal behavior and collecting visual data. The server acquires visual data from the imaging device in real time and processes the data using advanced image analysis algorithms. This allows for the analysis of animal behavioral characteristics and the detection of anomalies. For specific analysis, image processing libraries such as OpenCV can be used.

[0347] The server quickly sends detected anomaly information to the user's device. The device immediately notifies the user of the anomaly using push notification technology. Asynchronous communication is achieved by utilizing services such as Firebase Cloud Messaging. In addition, to take into account the user's stress and emotional state, the server uses an emotion analysis device to identify emotions from the user's voice and text. Machine learning tools such as TensorFlow can be used for emotion analysis.

[0348] Furthermore, based on the results of emotion analysis, customized notifications are provided according to the user's emotional state. This allows users to take appropriate action according to the animal's health condition. For example, for users experiencing significant stress, a notification message is generated that gently informs them of the situation and suggests specific countermeasures.

[0349] For example, if this system is used in a pet hotel, hotel staff can immediately identify any abnormal behavior in the animals being cared for and take necessary action. If a user is experiencing stress, the emotion analysis device will identify this and provide support with more helpful information.

[0350] By using the following example prompts, the emotion analyzer can perform highly accurate analysis. The notification wording can be adjusted based on scenarios such as "when the user is speaking" or "when the text message contains angry language." In this way, the present invention combines animal behavior monitoring with analysis of user emotional responses to provide an effective health management solution.

[0351] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0352] Step 1:

[0353] The server acquires visual data in real time from imaging devices installed within the animals' habitat. The input is a raw video stream from the imaging devices, which the server efficiently decodes and stores in memory. Specifically, it periodically captures frames of visual data and stores them in a high-speed cache. The output is video data in a format suitable for analysis.

[0354] Step 2:

[0355] The server processes the acquired video data using an analysis device. The input is the visual data obtained in step 1, and features such as the animal's movement speed, path, and posture are extracted through an image processing algorithm. Specifically, the OpenCV library is used to detect the outline of the animal in the video and calculate its motion vector. The output is numerical data indicating the animal's behavioral characteristics.

[0356] Step 3:

[0357] The server analyzes the behavioral characteristics data obtained in step 2 and detects anomalies. The input is numerical data obtained through feature extraction. A generative AI model is used to compare it with normal patterns and identify abnormal behavior. Specifically, an anomaly score is calculated to determine abnormal values, and a warning is generated if the threshold is exceeded. The output is a list of anomaly detection flags and anomaly details.

[0358] Step 4:

[0359] The server prepares to notify the terminal of the anomaly detection information. The input is the anomaly detection flag identified in step 3 and its details. A notification service (e.g., Firebase Cloud Messaging) is used to generate a push notification and send it to the user's terminal. Specifically, the notification message is formatted and the send request is processed asynchronously. The output is the notification message sent to the user's terminal.

[0360] Step 5:

[0361] The server uses an emotion analysis device to analyze the user's emotional state. Input consists of voice data and text messages provided by the user. Natural language processing (NLP) and machine learning models are used to calculate an emotion score. Specifically, speech recognition technology is used to convert speech to text, which is then analyzed by the emotion analysis algorithm. The output is the emotion evaluation score and its interpretation.

[0362] Step 6:

[0363] The server adjusts the notification content appropriately based on the user's sentiment rating score. The input is the sentiment score obtained in step 5. It generates prompts to change the wording and urgency of the notification, providing optimal information. Specifically, it selects a preset template and applies customizations according to the sentiment. The output is the adjusted notification message.

[0364] (Application Example 2)

[0365] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0366] The operation of the target objects, particularly industrial machinery and robots, requires rapid detection of abnormalities and appropriate countermeasures. However, current systems have challenges such as delayed anomaly detection and one-size-fits-all notifications to administrators, making it difficult to respond quickly and accurately in certain situations. Furthermore, methods for reducing the mental burden on administrators have not been sufficiently established.

[0367] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0368] In this invention, the server is a video acquisition device for capturing the movement of an object, and includes means installed within the movement range of the object, processing means for analyzing the captured video data in real time and analyzing the movement pattern of the object, and emotion analysis device means for analyzing the user's emotional state and adjusting the notification content. This enables rapid and accurate detection and notification of abnormalities, as well as the provision of flexible notification content according to the administrator's emotional state.

[0369] A "video acquisition device" is a device installed to film the movement of an object, and it records the movement within its range.

[0370] A "processing device" is a device that analyzes captured video data in real time and detects the movement patterns of an object.

[0371] A "notification device" is a device that immediately informs administrators of anomalies detected by an analysis device.

[0372] An "emotion analysis device" is a device that analyzes the user's emotional state and adjusts the content of notifications accordingly.

[0373] A "recording device" is a device used to record anomaly detection data and manage its history.

[0374] A "feedback device" is a device that has the function of adjusting the generated AI model based on feedback information in order to improve the accuracy of anomaly detection.

[0375] A "generative AI model" is an artificial intelligence model that can be adjusted to improve the accuracy of anomaly detection based on feedback.

[0376] To implement this invention, a camera is first required to identify the target objects, such as machinery and robots within a factory, and to monitor their operation in real time. This camera is installed as an AI camera and covers the operating range of the target objects. In addition, a server is required to process the captured video data and detect anomalies; this server acts as a processing unit.

[0377] The server uses OpenCV to analyze video data and analyzes behavioral patterns in real time. Based on these behavioral patterns, a generative AI model using TensorFlow detects anomalies and immediately notifies the administrator. Furthermore, the server uses an NLP library to analyze the administrator's emotional state and flexibly adjusts the notification content when the user is experiencing stress.

[0378] Furthermore, this system utilizes a recording device to record anomaly detection data, and a feedback device adjusts the generated AI model based on past anomaly detection records, thereby improving the accuracy of anomaly detection.

[0379] As a concrete example, the system detects abnormal vibrations occurring on a conveyor belt in a factory and uses this information to notify the manager of a potential overload condition. If the manager is under stress, the notification can be made in a gentler tone, such as, "There are some slight vibrations on the conveyor belt. No immediate action is required, but please check it periodically."

[0380] An example of a prompt message would be: "Use the AI ​​camera to monitor the robot's vibration patterns, and if an anomaly is detected, adjust the notification method according to the administrator's emotional state. Emotion analysis will help select the most appropriate notification format."

[0381] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0382] Step 1:

[0383] The AI ​​camera captures the movements of the target object, such as a machine or robot.

[0384] The input is real-time video of the object, and the output is recorded video data. The camera continuously acquires video and transmits the data to the server in real time.

[0385] Step 2:

[0386] The server uses OpenCV to analyze the video data and extract the motion patterns of the target object.

[0387] The input is the video data obtained in step 1, and the output is the motion pattern data. The server uses computer vision technology to extract motion features and convert the data into a format usable in the next processing step.

[0388] Step 3:

[0389] The server uses TensorFlow to perform anomaly detection using a generative AI model.

[0390] The input is the operation pattern data obtained in step 2, and the output is the anomaly detection result data. In this step, anomalies are identified by comparing them with known normal patterns. If an anomaly is detected, the information is sent to the next notification step.

[0391] Step 4:

[0392] The server uses an NLP library to analyze the administrator's emotional state.

[0393] The input is voice and text data from the administrator, and the output is emotional state data. Based on this emotional data, the server assesses the administrator's current stress level and prepares to generate appropriate notification content accordingly.

[0394] Step 5:

[0395] The server generates notification content tailored to the anomaly and the administrator's emotional state, and sends it to the administrator's terminal.

[0396] The input consists of the anomaly detection result data from step 3 and the emotional state data from step 4, and the output is a notification message to the administrator. This notification is flexibly adjusted according to the administrator's status and sent immediately.

[0397] Step 6:

[0398] The server saves the anomaly detection data to a recording device, and the AI ​​model is readjusted by a feedback device.

[0399] The input is anomaly detection result data, and the output is an updated generative AI model. In this step, the history is saved and used as feedback to train the AI ​​model, thereby improving the model's accuracy.

[0400] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0401] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0402] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0403] [Third Embodiment]

[0404] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0405] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0406] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0407] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0408] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0409] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0410] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0411] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0412] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0413] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0414] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0415] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0416] To implement this invention, video acquisition devices are installed within the animal's habitat, and a program that processes the video data obtained from these devices is run on a server. This system is designed to monitor the health status and behavioral abnormalities of animals and features real-time data processing and notification capabilities.

[0417] First, the terminal (video acquisition device) captures the animal's everyday behavior. This data is transmitted to a server in real time, where the behavioral patterns are analyzed. In particular, by capturing changes in the animal's movement speed, trajectory, and posture, abnormalities that deviate from normal behavioral patterns are detected.

[0418] If the server detects an anomaly, it immediately notifies the user terminal of the anomaly information. This allows the user to identify the anomaly early and take prompt action. In addition, the detected anomaly data and analysis results are stored in the server's recording device and can later be used as a database for health management.

[0419] Furthermore, in this invention, a feedback device plays a crucial role in continuously improving the AI ​​model. Based on feedback data obtained from the user, the model can be adjusted to improve the accuracy of anomaly detection. Through this process, the system can more accurately monitor the health status of animals and support the decisions of animal caretakers.

[0420] As a concrete example, consider a system for monitoring the walking patterns of cattle on a farm. AI cameras installed in the cattle grazing area capture the cattle's movements. A server analyzes this data and, if it detects any abnormalities such as limping, immediately sends a notification to the farm owner's terminal. This enables early diagnosis and treatment, improving farm efficiency while maintaining the health of the cattle.

[0421] By utilizing this system, it becomes possible to detect health abnormalities in animals early, reduce the burden on animal caretakers, and maintain the animals' health at an optimal level.

[0422] The following describes the processing flow.

[0423] Step 1:

[0424] The terminal (video acquisition device) is installed in the animal's living environment and continuously films the animal's behavior. The video data is transmitted to the server either in its original state or compressed.

[0425] Step 2:

[0426] The server receives the video data in real time and first performs preprocessing. Specifically, it extracts the characteristics of the animal's posture and movement frame by frame. Noise reduction and extraction of important feature points are performed during this process.

[0427] Step 3:

[0428] The server's AI model analyzes pre-processed feature data and compares it to the animal's normal behavior patterns. If an abnormal behavior pattern is detected, it is flagged as an anomaly, and a detailed evaluation is performed based on this flag.

[0429] Step 4:

[0430] The server immediately notifies the user terminal when an anomaly is detected. The notification includes information about the type of anomaly, the time it occurred, and its urgency.

[0431] Step 5:

[0432] Users receive notifications and, if necessary, contact a veterinarian or investigate the abnormal area. They can also consider and implement appropriate countermeasures.

[0433] Step 6:

[0434] The server records all anomaly detection data and response actions in a database. This historical data is used to analyze long-term trends in animal health and improve management strategies.

[0435] Step 7:

[0436] Users provide feedback to improve detection accuracy, and the server's AI model is updated based on this feedback. Through improvements to the AI ​​model, the anomaly detection capability is further enhanced, and the system becomes able to adapt to new situations.

[0437] (Example 1)

[0438] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0439] In modern animal husbandry and wildlife monitoring, the rapid and accurate detection of animal health conditions and behavioral abnormalities is crucial for reducing the burden on caretakers and ensuring animal safety. However, conventional systems lack real-time capabilities and accuracy, resulting in frequent false positives and detection delays. Furthermore, few systems currently incorporate automated model improvement using feedback.

[0440] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0441] In this invention, the server includes information processing means for analyzing captured image data in real time and analyzing the behavior patterns of organisms, communication means for immediately notifying of anomalies detected by the information processing means, and correction means for adjusting the learning model based on feedback to improve the accuracy of anomaly detection. This enables accurate anomaly detection in real time and continuous accuracy improvement using feedback.

[0442] "Image acquisition means" refers to a photographic device installed to record the movements of animals or living organisms, and is capable of continuously collecting data in the environment.

[0443] An "information processing device" is a processing device that analyzes acquired image data in real time and evaluates the movements and behaviors of animals, and has the function of analyzing data using AI models and algorithms.

[0444] "Communication means" refers to devices or systems that immediately notify users of abnormal information detected as a result of analysis, and that have the ability to transmit data using a network.

[0445] A "recording device" is a device used to store detected abnormal data and analysis results for later reference and analysis, and functions as a database.

[0446] A "correction mechanism" refers to a system or device that adjusts the AI ​​model based on feedback data to improve the accuracy of anomaly detection, and has a mechanism to improve the model's performance through automatic learning.

[0447] A "behavioral pattern" refers to a series of normal or abnormal behavioral patterns exhibited by animals or organisms, which can be identified through analysis.

[0448] A "learning model" is an artificial intelligence model used to analyze animal behavior and detect anomalies, and it is continuously adjusted to improve its accuracy based on data.

[0449] This invention is a system for monitoring the health status and behavioral abnormalities of animals in real time. The system consists of the following hardware and software.

[0450] First, the image acquisition device installed on the terminal records the animal's movements. This device consists of a camera capable of reliable recording even in outdoor environments. The camera can continuously capture the animal's posture, movement speed, and trajectory for 24 hours.

[0451] The acquired video data is transmitted to the server in real time. The server processes the acquired data using analysis software. Specifically, it uses the image analysis library "OpenCV" and the artificial intelligence framework "TensorFlow" to analyze the animal's behavior patterns. This makes it possible to determine whether the behavior deviates from normal patterns.

[0452] If an anomaly is detected, the server will immediately notify the user terminal. The user terminal consists of application software running on a smartphone or computer, and is designed to allow for quick verification and response upon receiving the notification.

[0453] Furthermore, it includes a correction mechanism that adjusts the AI ​​model based on feedback, allowing for continuous improvement of the model's accuracy. Users can contribute to system improvement by providing feedback on the analysis results and anomaly notifications provided.

[0454] As a concrete example, in a system that monitors the walking patterns of cows on a farm, video recorded by a terminal is sent to a server, and if an abnormality such as limping is detected, a notification is sent to the farm owner's smartphone. In this way, it becomes possible to detect illnesses early and provide appropriate treatment.

[0455] Another example of inputting prompts into a generative AI model is a text-based prompt such as, "List specific abnormalities in the animal's behavioral patterns." This prompt allows the AI ​​model to provide information on specific abnormal behaviors, thereby supporting the user's decision-making.

[0456] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0457] Step 1:

[0458] The terminal captures animal behavior in real time. The camera device monitors the habitat of the designated animal and continuously acquires image data. In this step, the input is video from the camera, and the output is video data. Specifically, the movement of the animal is captured through the camera lens and recorded as video.

[0459] Step 2:

[0460] The terminal transmits the acquired video data to the server in real time. In this process, the video data obtained in step 1 is used as input and transferred as a digital signal. As a result of this operation, the video data that reaches the server is obtained as output. Specifically, data transfer is performed using a network protocol.

[0461] Step 3:

[0462] The server analyzes the received video data using a generating AI model. The input is the video data received from step 2. The AI ​​model evaluates the animal's movement speed, trajectory, and posture changes, and performs data processing to identify anomalies from normal behavior patterns. The output is the analysis results if an anomaly is detected. The AI ​​model performs frame-by-frame analysis using "TensorFlow" and "OpenCV".

[0463] Step 4:

[0464] If the server detects an anomaly based on the analysis results, it immediately notifies the user terminal. The input uses the anomaly detection results obtained in step 3, and data conversion is performed to inform the user via the notification system. The output is an anomaly notification that is sent to the user terminal. Specifically, the anomaly is communicated via email or application notification.

[0465] Step 5:

[0466] The user reviews the system's analysis results through the received anomaly notification. The input for this process is the notification information from step 4, and the user reviews the current situation based on the displayed information and considers appropriate actions. The output is the user's judgment and the execution of countermeasures. Specifically, the user uses a smartphone or PC to review the details and decide on necessary actions such as physical inspection or treatment.

[0467] Step 6:

[0468] The server collects data to improve the accuracy of the AI ​​model through feedback. The input is user-provided feedback, which is used to adjust the AI ​​model. The output is an improved anomaly detection model. Specifically, model updates and tests are performed based on the feedback. This process enables the system to perform more accurate anomaly detection.

[0469] (Application Example 1)

[0470] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0471] Modern factories require the efficient and safe operation of industrial moving parts (such as robotic arms). However, malfunctions or abnormalities in these moving parts can lead to decreased productivity and safety threats, necessitating a rapid response. Conventional systems often rely on manual detection of anomalies and proposal of countermeasures, resulting in limitations in response speed and accuracy.

[0472] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0473] In this invention, the server is a video acquisition device for capturing the behavior of a moving object, and includes means installed at the work location of the moving object, processing means for analyzing the captured video data in real time and analyzing the behavior pattern of the moving object, and notification means for immediately notifying of any abnormalities detected by the analysis device. This enables rapid and accurate detection of abnormalities in the moving object and allows for the proposal of appropriate countermeasures, thereby improving productivity and ensuring safety.

[0474] "Operating device" refers to any mechanical device or system that performs a specific action, including industrial robot arms and automated machinery.

[0475] "Image acquisition equipment" is a general term for devices that include cameras and sensors for capturing the behavior of moving objects and collecting the resulting video data.

[0476] A "workplace" refers to a specific area or zone where an object performs its actions, such as a factory production line or a dedicated work area.

[0477] A "processing device" is a computer system that analyzes video data obtained from a video acquisition device and classifies and evaluates the behavior patterns of moving objects.

[0478] A "notification device" is a digital device or communication system that immediately transmits abnormalities detected by a processing unit to the user.

[0479] A "recording device" is a storage system that manages anomaly detection data and its history, and stores the data to enable later analysis and reference.

[0480] A "feedback device" is a platform that collects user feedback information and adjusts machine learning models with the aim of improving the detection accuracy of the system.

[0481] A "machine learning model" is a general term for algorithms and digital models used to learn from data and perform pattern recognition and anomaly detection.

[0482] To implement this invention, a system including the following components is required. First, a "video acquisition device" is installed at the work location of the moving object. The video acquisition device captures the behavior of the moving object in real time and transmits the video data to a server for processing. The server uses image analysis libraries such as Python and OpenCV to analyze the captured video data. In particular, the analysis device extracts parameters such as the movement speed, trajectory, and posture of the moving object and uses a machine learning model to identify normal behavior patterns and abnormal patterns.

[0483] If an anomaly is detected as a result of the analysis, the server quickly sends an alert to the user's terminal via a "notification device." The notification device is compatible with industrial monitoring systems and mobile devices, enabling immediate feedback. Furthermore, the anomaly detection data is stored in a "recording device" and used later for historical management. This data can be used as feedback to improve the accuracy of the analysis, and the server adaptively improves the machine learning model using a "feedback device."

[0484] A concrete example is a scenario where a robotic arm positioned on a factory production line performs packaging tasks. If the arm makes an unexpected movement and stops during normal operation, the abnormality is detected, and a warning is immediately sent to a terminal. This allows the operator to respond quickly and resume normal production activities.

[0485] Examples of prompt statements include the following:

[0486] "How can I monitor the behavior patterns of a robotic arm and issue an alert if it deviates from normal operation?"

[0487] The introduction of this system will significantly improve the production efficiency and safety of the factory.

[0488] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0489] Step 1:

[0490] The terminal captures the behavior of the working object from a video acquisition device placed at the work area where the object is performing its task. The captured video data is the input, and it is collected in video format. The terminal transmits this data to the server in real time.

[0491] Step 2:

[0492] The server uses Python and OpenCV to split the received video data into frames for processing. This split video data is used as input, and features such as the movement speed, trajectory, and orientation of the moving object are extracted from each frame. This results in a detailed dataset of the object's behavior.

[0493] Step 3:

[0494] The server uses a generative AI model to analyze the extracted features. The goal is to identify abnormal patterns by comparing them with pre-trained normal operating patterns. The input is the feature data from step 2, and the output is whether or not an anomaly was detected and the type of anomaly.

[0495] Step 4:

[0496] If an anomaly is detected, the server immediately sends an alert to the user's terminal via a notification device. Here, the presence or absence of an anomaly is the input, and a warning message to the user is output. For example, a specific message such as "An anomaly has been detected in the robot arm" might be displayed.

[0497] Step 5:

[0498] The server stores detected anomaly data in a recording device. This data serves as historical data for long-term analysis and feedback. The input is detailed data of the anomaly detection, and the output is the stored database entry.

[0499] Step 6:

[0500] Users can provide evaluations and additional information regarding detection results through a feedback device. This feedback is input for system improvement, and the server uses it to adjust the machine learning model and improve the accuracy of anomaly detection. As a result, a more powerful model is output.

[0501] Through the above process, it becomes possible to monitor the behavior of the moving object in detail and to quickly detect and notify of any abnormalities.

[0502] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0503] To implement this invention, a video acquisition device is installed within the animal's habitat, and the video data obtained from the device is processed on a server. A processing device that analyzes the animal's behavior patterns detects abnormalities based on its movement speed, trajectory, and posture, and quickly notifies the user's terminal of this information. This allows the user to respond early to changes in the animal's health condition.

[0504] Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, thereby adjusting notification methods and recommendations according to the user's emotional state. Specifically, the server uses the emotion engine to recognize the user's emotional state from their voice and text. Based on the specific emotion, it then adjusts the wording and urgency level of notifications to provide the user with the most appropriate information.

[0505] For example, users identified as being under significant stress will receive notifications in a softer tone and receive further support when suggesting specific solutions. The emotion engine continues to learn from user feedback, which is also used to fine-tune the AI ​​model to improve the accuracy of notifications.

[0506] As a concrete example of this invention, consider its implementation in a pet hotel. An AI camera captures the behavior of dogs left at the pet hotel, and if the server detects abnormal behavior, a user acting as a hotel staff member receives a notification on their smartphone. If the user is busy and stressed, the server recognizes their emotions through an emotion engine and provides detailed support along with further notifications.

[0507] This system not only quickly detects abnormalities in animals, but also reduces the user's mental burden and supports efficient health management.

[0508] The following describes the processing flow.

[0509] Step 1:

[0510] The terminal (video acquisition device) is placed in the animal's living environment and continuously acquires the animal's behavior. This video data is transmitted to a server via the network.

[0511] Step 2:

[0512] The server preprocesses the received video data and extracts the animal's movement characteristics. This includes frame-by-frame posture recognition and movement pattern analysis, enabling the detection of abnormal movements.

[0513] Step 3:

[0514] The AI ​​model on the server analyzes animal behavior based on pre-processed data, distinguishing between normal and abnormal behavior. During this process, a behavioral prediction algorithm is used to assess the animal's health status.

[0515] Step 4:

[0516] If abnormal behavior is detected, the server will immediately notify the user terminal of the details of the anomaly. The notification will include the specific type of anomaly, the time it occurred, and the recommended next steps.

[0517] Step 5:

[0518] The server uses an emotion engine to analyze the user's emotional state. It recognizes emotions from the user's voice and text input, adjusts notification content based on the results, and shares information appropriately.

[0519] Step 6:

[0520] Users receive notifications and check the situation. Based on the notifications, which are adjusted by the emotion engine, they can consider and implement ways to alleviate tension and specific countermeasures.

[0521] Step 7:

[0522] The server records detected anomalies and user feedback in a database. This information is used to improve long-term animal health management and enhance the accuracy of AI models.

[0523] Step 8:

[0524] Based on user feedback, the server's AI and emotion engine undergo regular adjustments to improve the quality of anomaly detection and user notifications.

[0525] (Example 2)

[0526] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0527] Monitoring animal behavior and detecting abnormalities early is crucial for animal health management. However, conventional monitoring systems have problems such as difficulty in analyzing animal behavioral characteristics in detail and in not being able to quickly notify users of abnormalities. Furthermore, the information provided does not take into account the emotional state of the user receiving the notification, making it difficult to take appropriate measures.

[0528] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0529] In this invention, the server is an imaging device for capturing animal behavior, and includes means for being installed in the area of ​​the organism; analysis device means for immediately analyzing the captured visual data and analyzing the behavioral characteristics of the organism; and emotion analysis device means for analyzing the user's emotional information and adjusting the notification method. This enables detailed analysis of the animal's behavioral characteristics and rapid detection of abnormalities, and further enables the provision of accurate notifications and support according to the user's emotional state.

[0530] An "imaging device" is a device that records the behavior of animals and is installed in areas where living organisms reside.

[0531] "Visual data" refers to video information acquired by an imaging device.

[0532] An "analysis device" is a device that instantly analyzes captured visual data to identify the behavioral characteristics of organisms.

[0533] A "notification device" is a device that quickly informs the user of abnormal information detected by an analysis device.

[0534] A "storage device" is a device used to store anomaly detection data and maintain records.

[0535] A "regulating device" is a device that adjusts a machine learning model based on feedback in order to improve the accuracy of anomaly detection.

[0536] An "emotion analysis device" is a device that analyzes the user's emotional information and adjusts the method of communication accordingly.

[0537] "Behavioral characteristics of an organism" refers to its usual behavioral patterns, including movement speed, route, and posture.

[0538] To implement this invention, it is necessary to install an imaging device in an area where animals live. This imaging device has the function of recording animal behavior and collecting visual data. The server acquires visual data from the imaging device in real time and processes the data using advanced image analysis algorithms. This allows for the analysis of animal behavioral characteristics and the detection of anomalies. For specific analysis, image processing libraries such as OpenCV can be used.

[0539] The server quickly sends detected anomaly information to the user's device. The device immediately notifies the user of the anomaly using push notification technology. Asynchronous communication is achieved by utilizing services such as Firebase Cloud Messaging. In addition, to take into account the user's stress and emotional state, the server uses an emotion analysis device to identify emotions from the user's voice and text. Machine learning tools such as TensorFlow can be used for emotion analysis.

[0540] Furthermore, based on the results of emotion analysis, customized notifications are provided according to the user's emotional state. This allows users to take appropriate action according to the animal's health condition. For example, for users experiencing significant stress, a notification message is generated that gently informs them of the situation and suggests specific countermeasures.

[0541] For example, if this system is used in a pet hotel, hotel staff can immediately identify any abnormal behavior in the animals being cared for and take necessary action. If a user is experiencing stress, the emotion analysis device will identify this and provide support with more helpful information.

[0542] By using the following example prompts, the emotion analyzer can perform highly accurate analysis. The notification wording can be adjusted based on scenarios such as "when the user is speaking" or "when the text message contains angry language." In this way, the present invention combines animal behavior monitoring with analysis of user emotional responses to provide an effective health management solution.

[0543] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0544] Step 1:

[0545] The server acquires visual data in real time from imaging devices installed within the animals' habitat. The input is a raw video stream from the imaging devices, which the server efficiently decodes and stores in memory. Specifically, it periodically captures frames of visual data and stores them in a high-speed cache. The output is video data in a format suitable for analysis.

[0546] Step 2:

[0547] The server processes the acquired video data using an analysis device. The input is the visual data obtained in step 1, and features such as the animal's movement speed, path, and posture are extracted through an image processing algorithm. Specifically, the OpenCV library is used to detect the outline of the animal in the video and calculate its motion vector. The output is numerical data indicating the animal's behavioral characteristics.

[0548] Step 3:

[0549] The server analyzes the behavioral characteristics data obtained in step 2 and detects anomalies. The input is numerical data obtained through feature extraction. A generative AI model is used to compare it with normal patterns and identify abnormal behavior. Specifically, an anomaly score is calculated to determine abnormal values, and a warning is generated if the threshold is exceeded. The output is a list of anomaly detection flags and anomaly details.

[0550] Step 4:

[0551] The server prepares to notify the terminal of the anomaly detection information. The input is the anomaly detection flag identified in step 3 and its details. A notification service (e.g., Firebase Cloud Messaging) is used to generate a push notification and send it to the user's terminal. Specifically, the notification message is formatted and the send request is processed asynchronously. The output is the notification message sent to the user's terminal.

[0552] Step 5:

[0553] The server uses an emotion analysis device to analyze the user's emotional state. Input consists of voice data and text messages provided by the user. Natural language processing (NLP) and machine learning models are used to calculate an emotion score. Specifically, speech recognition technology is used to convert speech to text, which is then analyzed by the emotion analysis algorithm. The output is the emotion evaluation score and its interpretation.

[0554] Step 6:

[0555] The server adjusts the notification content appropriately based on the user's sentiment rating score. The input is the sentiment score obtained in step 5. It generates prompts to change the wording and urgency of the notification, providing optimal information. Specifically, it selects a preset template and applies customizations according to the sentiment. The output is the adjusted notification message.

[0556] (Application Example 2)

[0557] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0558] The operation of the target objects, particularly industrial machinery and robots, requires rapid detection of abnormalities and appropriate countermeasures. However, current systems have challenges such as delayed anomaly detection and one-size-fits-all notifications to administrators, making it difficult to respond quickly and accurately in certain situations. Furthermore, methods for reducing the mental burden on administrators have not been sufficiently established.

[0559] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0560] In this invention, the server is a video acquisition device for capturing the movement of an object, and includes means installed within the movement range of the object, processing means for analyzing the captured video data in real time and analyzing the movement pattern of the object, and emotion analysis device means for analyzing the user's emotional state and adjusting the notification content. This enables rapid and accurate detection and notification of abnormalities, as well as the provision of flexible notification content according to the administrator's emotional state.

[0561] A "video acquisition device" is a device installed to film the movement of an object, and it records the movement within its range.

[0562] A "processing device" is a device that analyzes captured video data in real time and detects the movement patterns of an object.

[0563] A "notification device" is a device that immediately informs administrators of anomalies detected by an analysis device.

[0564] An "emotion analysis device" is a device that analyzes the user's emotional state and adjusts the content of notifications accordingly.

[0565] A "recording device" is a device used to record anomaly detection data and manage its history.

[0566] A "feedback device" is a device that has the function of adjusting the generated AI model based on feedback information in order to improve the accuracy of anomaly detection.

[0567] A "generative AI model" is an artificial intelligence model that can be adjusted to improve the accuracy of anomaly detection based on feedback.

[0568] To implement this invention, a camera is first required to identify the target objects, such as machinery and robots within a factory, and to monitor their operation in real time. This camera is installed as an AI camera and covers the operating range of the target objects. In addition, a server is required to process the captured video data and detect anomalies; this server acts as a processing unit.

[0569] The server uses OpenCV to analyze video data and analyzes behavioral patterns in real time. Based on these behavioral patterns, a generative AI model using TensorFlow detects anomalies and immediately notifies the administrator. Furthermore, the server uses an NLP library to analyze the administrator's emotional state and flexibly adjusts the notification content when the user is experiencing stress.

[0570] Furthermore, this system utilizes a recording device to record anomaly detection data, and a feedback device adjusts the generated AI model based on past anomaly detection records, thereby improving the accuracy of anomaly detection.

[0571] As a concrete example, the system detects abnormal vibrations occurring on a conveyor belt in a factory and uses this information to notify the manager of a potential overload condition. If the manager is under stress, the notification can be made in a gentler tone, such as, "There are some slight vibrations on the conveyor belt. No immediate action is required, but please check it periodically."

[0572] An example of a prompt message would be: "Use the AI ​​camera to monitor the robot's vibration patterns, and if an anomaly is detected, adjust the notification method according to the administrator's emotional state. Emotion analysis will help select the most appropriate notification format."

[0573] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0574] Step 1:

[0575] The AI ​​camera captures the movements of the target object, such as a machine or robot.

[0576] The input is real-time video of the object, and the output is recorded video data. The camera continuously acquires video and transmits the data to the server in real time.

[0577] Step 2:

[0578] The server uses OpenCV to analyze the video data and extract the motion patterns of the target object.

[0579] The input is the video data obtained in step 1, and the output is the motion pattern data. The server uses computer vision technology to extract motion features and convert the data into a format usable in the next processing step.

[0580] Step 3:

[0581] The server uses TensorFlow to perform anomaly detection using a generative AI model.

[0582] The input is the operation pattern data obtained in step 2, and the output is the anomaly detection result data. In this step, anomalies are identified by comparing them with known normal patterns. If an anomaly is detected, the information is sent to the next notification step.

[0583] Step 4:

[0584] The server uses an NLP library to analyze the administrator's emotional state.

[0585] The input is voice and text data from the administrator, and the output is emotional state data. Based on this emotional data, the server assesses the administrator's current stress level and prepares to generate appropriate notification content accordingly.

[0586] Step 5:

[0587] The server generates notification content tailored to the anomaly and the administrator's emotional state, and sends it to the administrator's terminal.

[0588] The input consists of the anomaly detection result data from step 3 and the emotional state data from step 4, and the output is a notification message to the administrator. This notification is flexibly adjusted according to the administrator's status and sent immediately.

[0589] Step 6:

[0590] The server saves the anomaly detection data to a recording device, and the AI ​​model is readjusted by a feedback device.

[0591] The input is anomaly detection result data, and the output is an updated generative AI model. In this step, the history is saved and used as feedback to train the AI ​​model, thereby improving the model's accuracy.

[0592] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0593] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0594] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0595] [Fourth Embodiment]

[0596] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0597] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0598] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0599] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0600] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0601] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0602] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0603] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0604] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0605] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0606] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0607] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0608] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0609] To implement this invention, video acquisition devices are installed within the animal's habitat, and a program that processes the video data obtained from these devices is run on a server. This system is designed to monitor the health status and behavioral abnormalities of animals and features real-time data processing and notification capabilities.

[0610] First, the terminal (video acquisition device) captures the animal's everyday behavior. This data is transmitted to a server in real time, where the behavioral patterns are analyzed. In particular, by capturing changes in the animal's movement speed, trajectory, and posture, abnormalities that deviate from normal behavioral patterns are detected.

[0611] If the server detects an anomaly, it immediately notifies the user terminal of the anomaly information. This allows the user to identify the anomaly early and take prompt action. In addition, the detected anomaly data and analysis results are stored in the server's recording device and can later be used as a database for health management.

[0612] Furthermore, in this invention, a feedback device plays a crucial role in continuously improving the AI ​​model. Based on feedback data obtained from the user, the model can be adjusted to improve the accuracy of anomaly detection. Through this process, the system can more accurately monitor the health status of animals and support the decisions of animal caretakers.

[0613] As a concrete example, consider a system for monitoring the walking patterns of cattle on a farm. AI cameras installed in the cattle grazing area capture the cattle's movements. A server analyzes this data and, if it detects any abnormalities such as limping, immediately sends a notification to the farm owner's terminal. This enables early diagnosis and treatment, improving farm efficiency while maintaining the health of the cattle.

[0614] By utilizing this system, it becomes possible to detect health abnormalities in animals early, reduce the burden on animal caretakers, and maintain the animals' health at an optimal level.

[0615] The following describes the processing flow.

[0616] Step 1:

[0617] The terminal (video acquisition device) is installed in the animal's living environment and continuously films the animal's behavior. The video data is transmitted to the server either in its original state or compressed.

[0618] Step 2:

[0619] The server receives the video data in real time and first performs preprocessing. Specifically, it extracts the characteristics of the animal's posture and movement frame by frame. Noise reduction and extraction of important feature points are performed during this process.

[0620] Step 3:

[0621] The server's AI model analyzes pre-processed feature data and compares it to the animal's normal behavior patterns. If an abnormal behavior pattern is detected, it is flagged as an anomaly, and a detailed evaluation is performed based on this flag.

[0622] Step 4:

[0623] The server immediately notifies the user terminal when an anomaly is detected. The notification includes information about the type of anomaly, the time it occurred, and its urgency.

[0624] Step 5:

[0625] Users receive notifications and, if necessary, contact a veterinarian or investigate the abnormal area. They can also consider and implement appropriate countermeasures.

[0626] Step 6:

[0627] The server records all anomaly detection data and response actions in a database. This historical data is used to analyze long-term trends in animal health and improve management strategies.

[0628] Step 7:

[0629] Users provide feedback to improve detection accuracy, and the server's AI model is updated based on this feedback. Through improvements to the AI ​​model, the anomaly detection capability is further enhanced, and the system becomes able to adapt to new situations.

[0630] (Example 1)

[0631] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0632] In modern animal husbandry and wildlife monitoring, the rapid and accurate detection of animal health conditions and behavioral abnormalities is crucial for reducing the burden on caretakers and ensuring animal safety. However, conventional systems lack real-time capabilities and accuracy, resulting in frequent false positives and detection delays. Furthermore, few systems currently incorporate automated model improvement using feedback.

[0633] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0634] In this invention, the server includes information processing means for analyzing captured image data in real time and analyzing the behavior patterns of organisms, communication means for immediately notifying of anomalies detected by the information processing means, and correction means for adjusting the learning model based on feedback to improve the accuracy of anomaly detection. This enables accurate anomaly detection in real time and continuous accuracy improvement using feedback.

[0635] "Image acquisition means" refers to a photographic device installed to record the movements of animals or living organisms, and is capable of continuously collecting data in the environment.

[0636] An "information processing device" is a processing device that analyzes acquired image data in real time and evaluates the movements and behaviors of animals, and has the function of analyzing data using AI models and algorithms.

[0637] "Communication means" refers to devices or systems that immediately notify users of abnormal information detected as a result of analysis, and that have the ability to transmit data using a network.

[0638] A "recording device" is a device used to store detected abnormal data and analysis results for later reference and analysis, and functions as a database.

[0639] A "correction mechanism" refers to a system or device that adjusts the AI ​​model based on feedback data to improve the accuracy of anomaly detection, and has a mechanism to improve the model's performance through automatic learning.

[0640] A "behavioral pattern" refers to a series of normal or abnormal behavioral patterns exhibited by animals or organisms, which can be identified through analysis.

[0641] A "learning model" is an artificial intelligence model used to analyze animal behavior and detect anomalies, and it is continuously adjusted to improve its accuracy based on data.

[0642] This invention is a system for monitoring the health status and behavioral abnormalities of animals in real time. The system consists of the following hardware and software.

[0643] First, the image acquisition device installed on the terminal records the animal's movements. This device consists of a camera capable of reliable recording even in outdoor environments. The camera can continuously capture the animal's posture, movement speed, and trajectory for 24 hours.

[0644] The acquired video data is transmitted to the server in real time. The server processes the acquired data using analysis software. Specifically, it uses the image analysis library "OpenCV" and the artificial intelligence framework "TensorFlow" to analyze the animal's behavior patterns. This makes it possible to determine whether the behavior deviates from normal patterns.

[0645] If an anomaly is detected, the server will immediately notify the user terminal. The user terminal consists of application software running on a smartphone or computer, and is designed to allow for quick verification and response upon receiving the notification.

[0646] Furthermore, it includes a correction mechanism that adjusts the AI ​​model based on feedback, allowing for continuous improvement of the model's accuracy. Users can contribute to system improvement by providing feedback on the analysis results and anomaly notifications provided.

[0647] As a concrete example, in a system that monitors the walking patterns of cows on a farm, video recorded by a terminal is sent to a server, and if an abnormality such as limping is detected, a notification is sent to the farm owner's smartphone. In this way, it becomes possible to detect illnesses early and provide appropriate treatment.

[0648] Another example of inputting prompts into a generative AI model is a text-based prompt such as, "List specific abnormalities in the animal's behavioral patterns." This prompt allows the AI ​​model to provide information on specific abnormal behaviors, thereby supporting the user's decision-making.

[0649] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0650] Step 1:

[0651] The terminal captures animal behavior in real time. The camera device monitors the habitat of the designated animal and continuously acquires image data. In this step, the input is video from the camera, and the output is video data. Specifically, the movement of the animal is captured through the camera lens and recorded as video.

[0652] Step 2:

[0653] The terminal transmits the acquired video data to the server in real time. In this process, the video data obtained in step 1 is used as input and transferred as a digital signal. As a result of this operation, the video data that reaches the server is obtained as output. Specifically, data transfer is performed using a network protocol.

[0654] Step 3:

[0655] The server analyzes the received video data using a generating AI model. The input is the video data received from step 2. The AI ​​model evaluates the animal's movement speed, trajectory, and posture changes, and performs data processing to identify anomalies from normal behavior patterns. The output is the analysis results if an anomaly is detected. The AI ​​model performs frame-by-frame analysis using "TensorFlow" and "OpenCV".

[0656] Step 4:

[0657] If the server detects an anomaly based on the analysis results, it immediately notifies the user terminal. The input uses the anomaly detection results obtained in step 3, and data conversion is performed to inform the user via the notification system. The output is an anomaly notification that is sent to the user terminal. Specifically, the anomaly is communicated via email or application notification.

[0658] Step 5:

[0659] The user reviews the system's analysis results through the received anomaly notification. The input for this process is the notification information from step 4, and the user reviews the current situation based on the displayed information and considers appropriate actions. The output is the user's judgment and the execution of countermeasures. Specifically, the user uses a smartphone or PC to review the details and decide on necessary actions such as physical inspection or treatment.

[0660] Step 6:

[0661] The server collects data to improve the accuracy of the AI ​​model through feedback. The input is user-provided feedback, which is used to adjust the AI ​​model. The output is an improved anomaly detection model. Specifically, model updates and tests are performed based on the feedback. This process enables the system to perform more accurate anomaly detection.

[0662] (Application Example 1)

[0663] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0664] Modern factories require the efficient and safe operation of industrial moving parts (such as robotic arms). However, malfunctions or abnormalities in these moving parts can lead to decreased productivity and safety threats, necessitating a rapid response. Conventional systems often rely on manual detection of anomalies and proposal of countermeasures, resulting in limitations in response speed and accuracy.

[0665] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0666] In this invention, the server is a video acquisition device for capturing the behavior of a moving object, and includes means installed at the work location of the moving object, processing means for analyzing the captured video data in real time and analyzing the behavior pattern of the moving object, and notification means for immediately notifying of any abnormalities detected by the analysis device. This enables rapid and accurate detection of abnormalities in the moving object and allows for the proposal of appropriate countermeasures, thereby improving productivity and ensuring safety.

[0667] "Operating device" refers to any mechanical device or system that performs a specific action, including industrial robot arms and automated machinery.

[0668] "Image acquisition equipment" is a general term for devices that include cameras and sensors for capturing the behavior of moving objects and collecting the resulting video data.

[0669] A "workplace" refers to a specific area or zone where an object performs its actions, such as a factory production line or a dedicated work area.

[0670] A "processing device" is a computer system that analyzes video data obtained from a video acquisition device and classifies and evaluates the behavior patterns of moving objects.

[0671] A "notification device" is a digital device or communication system that immediately transmits abnormalities detected by a processing unit to the user.

[0672] A "recording device" is a storage system that manages anomaly detection data and its history, and stores the data to enable later analysis and reference.

[0673] A "feedback device" is a platform that collects user feedback information and adjusts machine learning models with the aim of improving the detection accuracy of the system.

[0674] A "machine learning model" is a general term for algorithms and digital models used to learn from data and perform pattern recognition and anomaly detection.

[0675] To implement this invention, a system including the following components is required. First, a "video acquisition device" is installed at the work location of the moving object. The video acquisition device captures the behavior of the moving object in real time and transmits the video data to a server for processing. The server uses image analysis libraries such as Python and OpenCV to analyze the captured video data. In particular, the analysis device extracts parameters such as the movement speed, trajectory, and posture of the moving object and uses a machine learning model to identify normal behavior patterns and abnormal patterns.

[0676] If an anomaly is detected as a result of the analysis, the server quickly sends an alert to the user's terminal via a "notification device." The notification device is compatible with industrial monitoring systems and mobile devices, enabling immediate feedback. Furthermore, the anomaly detection data is stored in a "recording device" and used later for historical management. This data can be used as feedback to improve the accuracy of the analysis, and the server adaptively improves the machine learning model using a "feedback device."

[0677] A concrete example is a scenario where a robotic arm positioned on a factory production line performs packaging tasks. If the arm makes an unexpected movement and stops during normal operation, the abnormality is detected, and a warning is immediately sent to a terminal. This allows the operator to respond quickly and resume normal production activities.

[0678] Examples of prompt statements include the following:

[0679] "How can I monitor the behavior patterns of a robotic arm and issue an alert if it deviates from normal operation?"

[0680] The introduction of this system will significantly improve the production efficiency and safety of the factory.

[0681] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0682] Step 1:

[0683] The terminal captures the behavior of the working object from a video acquisition device placed at the work area where the object is performing its task. The captured video data is the input, and it is collected in video format. The terminal transmits this data to the server in real time.

[0684] Step 2:

[0685] The server uses Python and OpenCV to split the received video data into frames for processing. This split video data is used as input, and features such as the movement speed, trajectory, and orientation of the moving object are extracted from each frame. This results in a detailed dataset of the object's behavior.

[0686] Step 3:

[0687] The server uses a generative AI model to analyze the extracted features. The goal is to identify abnormal patterns by comparing them with pre-trained normal operating patterns. The input is the feature data from step 2, and the output is whether or not an anomaly was detected and the type of anomaly.

[0688] Step 4:

[0689] If an anomaly is detected, the server immediately sends an alert to the user's terminal via a notification device. Here, the presence or absence of an anomaly is the input, and a warning message to the user is output. For example, a specific message such as "An anomaly has been detected in the robot arm" might be displayed.

[0690] Step 5:

[0691] The server stores detected anomaly data in a recording device. This data serves as historical data for long-term analysis and feedback. The input is detailed data of the anomaly detection, and the output is the stored database entry.

[0692] Step 6:

[0693] Users can provide evaluations and additional information regarding detection results through a feedback device. This feedback is input for system improvement, and the server uses it to adjust the machine learning model and improve the accuracy of anomaly detection. As a result, a more powerful model is output.

[0694] Through the above process, it becomes possible to monitor the behavior of the moving object in detail and to quickly detect and notify of any abnormalities.

[0695] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0696] To implement this invention, a video acquisition device is installed within the animal's habitat, and the video data obtained from the device is processed on a server. A processing device that analyzes the animal's behavior patterns detects abnormalities based on its movement speed, trajectory, and posture, and quickly notifies the user's terminal of this information. This allows the user to respond early to changes in the animal's health condition.

[0697] Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, thereby adjusting notification methods and recommendations according to the user's emotional state. Specifically, the server uses the emotion engine to recognize the user's emotional state from their voice and text. Based on the specific emotion, it then adjusts the wording and urgency level of notifications to provide the user with the most appropriate information.

[0698] For example, users identified as being under significant stress will receive notifications in a softer tone and receive further support when suggesting specific solutions. The emotion engine continues to learn from user feedback, which is also used to fine-tune the AI ​​model to improve the accuracy of notifications.

[0699] As a concrete example of this invention, consider its implementation in a pet hotel. An AI camera captures the behavior of dogs left at the pet hotel, and if the server detects abnormal behavior, a user acting as a hotel staff member receives a notification on their smartphone. If the user is busy and stressed, the server recognizes their emotions through an emotion engine and provides detailed support along with further notifications.

[0700] This system not only quickly detects abnormalities in animals, but also reduces the user's mental burden and supports efficient health management.

[0701] The following describes the processing flow.

[0702] Step 1:

[0703] The terminal (video acquisition device) is placed in the animal's living environment and continuously acquires the animal's behavior. This video data is transmitted to a server via the network.

[0704] Step 2:

[0705] The server preprocesses the received video data and extracts the animal's movement characteristics. This includes frame-by-frame posture recognition and movement pattern analysis, enabling the detection of abnormal movements.

[0706] Step 3:

[0707] The AI ​​model on the server analyzes animal behavior based on pre-processed data, distinguishing between normal and abnormal behavior. During this process, a behavioral prediction algorithm is used to assess the animal's health status.

[0708] Step 4:

[0709] If abnormal behavior is detected, the server will immediately notify the user terminal of the details of the anomaly. The notification will include the specific type of anomaly, the time it occurred, and the recommended next steps.

[0710] Step 5:

[0711] The server uses an emotion engine to analyze the user's emotional state. It recognizes emotions from the user's voice and text input, adjusts notification content based on the results, and shares information appropriately.

[0712] Step 6:

[0713] Users receive notifications and check the situation. Based on the notifications, which are adjusted by the emotion engine, they can consider and implement ways to alleviate tension and specific countermeasures.

[0714] Step 7:

[0715] The server records detected anomalies and user feedback in a database. This information is used to improve long-term animal health management and enhance the accuracy of AI models.

[0716] Step 8:

[0717] Based on user feedback, the server's AI and emotion engine undergo regular adjustments to improve the quality of anomaly detection and user notifications.

[0718] (Example 2)

[0719] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0720] Monitoring animal behavior and detecting abnormalities early is crucial for animal health management. However, conventional monitoring systems have problems such as difficulty in analyzing animal behavioral characteristics in detail and in not being able to quickly notify users of abnormalities. Furthermore, the information provided does not take into account the emotional state of the user receiving the notification, making it difficult to take appropriate measures.

[0721] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0722] In this invention, the server is an imaging device for capturing animal behavior, and includes means for being installed in the area of ​​the organism; analysis device means for immediately analyzing the captured visual data and analyzing the behavioral characteristics of the organism; and emotion analysis device means for analyzing the user's emotional information and adjusting the notification method. This enables detailed analysis of the animal's behavioral characteristics and rapid detection of abnormalities, and further enables the provision of accurate notifications and support according to the user's emotional state.

[0723] An "imaging device" is a device that records the behavior of animals and is installed in areas where living organisms reside.

[0724] "Visual data" refers to video information acquired by an imaging device.

[0725] An "analysis device" is a device that instantly analyzes captured visual data to identify the behavioral characteristics of organisms.

[0726] A "notification device" is a device that quickly informs the user of abnormal information detected by an analysis device.

[0727] A "storage device" is a device used to store anomaly detection data and maintain records.

[0728] A "regulating device" is a device that adjusts a machine learning model based on feedback in order to improve the accuracy of anomaly detection.

[0729] An "emotion analysis device" is a device that analyzes the user's emotional information and adjusts the method of communication accordingly.

[0730] "Behavioral characteristics of an organism" refers to its usual behavioral patterns, including movement speed, route, and posture.

[0731] To implement this invention, it is necessary to install an imaging device in an area where animals live. This imaging device has the function of recording animal behavior and collecting visual data. The server acquires visual data from the imaging device in real time and processes the data using advanced image analysis algorithms. This allows for the analysis of animal behavioral characteristics and the detection of anomalies. For specific analysis, image processing libraries such as OpenCV can be used.

[0732] The server quickly sends detected anomaly information to the user's device. The device immediately notifies the user of the anomaly using push notification technology. Asynchronous communication is achieved by utilizing services such as Firebase Cloud Messaging. In addition, to take into account the user's stress and emotional state, the server uses an emotion analysis device to identify emotions from the user's voice and text. Machine learning tools such as TensorFlow can be used for emotion analysis.

[0733] Furthermore, based on the results of emotion analysis, customized notifications are provided according to the user's emotional state. This allows users to take appropriate action according to the animal's health condition. For example, for users experiencing significant stress, a notification message is generated that gently informs them of the situation and suggests specific countermeasures.

[0734] For example, if this system is used in a pet hotel, hotel staff can immediately identify any abnormal behavior in the animals being cared for and take necessary action. If a user is experiencing stress, the emotion analysis device will identify this and provide support with more helpful information.

[0735] By using the following example prompts, the emotion analyzer can perform highly accurate analysis. The notification wording can be adjusted based on scenarios such as "when the user is speaking" or "when the text message contains angry language." In this way, the present invention combines animal behavior monitoring with analysis of user emotional responses to provide an effective health management solution.

[0736] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0737] Step 1:

[0738] The server acquires visual data in real time from imaging devices installed within the animals' habitat. The input is a raw video stream from the imaging devices, which the server efficiently decodes and stores in memory. Specifically, it periodically captures frames of visual data and stores them in a high-speed cache. The output is video data in a format suitable for analysis.

[0739] Step 2:

[0740] The server processes the acquired video data using an analysis device. The input is the visual data obtained in step 1, and features such as the animal's movement speed, path, and posture are extracted through an image processing algorithm. Specifically, the OpenCV library is used to detect the outline of the animal in the video and calculate its motion vector. The output is numerical data indicating the animal's behavioral characteristics.

[0741] Step 3:

[0742] The server analyzes the behavioral characteristics data obtained in step 2 and detects anomalies. The input is numerical data obtained through feature extraction. A generative AI model is used to compare it with normal patterns and identify abnormal behavior. Specifically, an anomaly score is calculated to determine abnormal values, and a warning is generated if the threshold is exceeded. The output is a list of anomaly detection flags and anomaly details.

[0743] Step 4:

[0744] The server prepares to notify the terminal of the anomaly detection information. The input is the anomaly detection flag identified in step 3 and its details. A notification service (e.g., Firebase Cloud Messaging) is used to generate a push notification and send it to the user's terminal. Specifically, the notification message is formatted and the send request is processed asynchronously. The output is the notification message sent to the user's terminal.

[0745] Step 5:

[0746] The server uses an emotion analysis device to analyze the user's emotional state. Input consists of voice data and text messages provided by the user. Natural language processing (NLP) and machine learning models are used to calculate an emotion score. Specifically, speech recognition technology is used to convert speech to text, which is then analyzed by the emotion analysis algorithm. The output is the emotion evaluation score and its interpretation.

[0747] Step 6:

[0748] The server adjusts the notification content appropriately based on the user's sentiment rating score. The input is the sentiment score obtained in step 5. It generates prompts to change the wording and urgency of the notification, providing optimal information. Specifically, it selects a preset template and applies customizations according to the sentiment. The output is the adjusted notification message.

[0749] (Application Example 2)

[0750] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0751] The operation of the target objects, particularly industrial machinery and robots, requires rapid detection of abnormalities and appropriate countermeasures. However, current systems have challenges such as delayed anomaly detection and one-size-fits-all notifications to administrators, making it difficult to respond quickly and accurately in certain situations. Furthermore, methods for reducing the mental burden on administrators have not been sufficiently established.

[0752] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0753] In this invention, the server is a video acquisition device for capturing the movement of an object, and includes means installed within the movement range of the object, processing means for analyzing the captured video data in real time and analyzing the movement pattern of the object, and emotion analysis device means for analyzing the user's emotional state and adjusting the notification content. This enables rapid and accurate detection and notification of abnormalities, as well as the provision of flexible notification content according to the administrator's emotional state.

[0754] A "video acquisition device" is a device installed to film the movement of an object, and it records the movement within its range.

[0755] A "processing device" is a device that analyzes captured video data in real time and detects the movement patterns of an object.

[0756] A "notification device" is a device that immediately informs administrators of anomalies detected by an analysis device.

[0757] An "emotion analysis device" is a device that analyzes the user's emotional state and adjusts the content of notifications accordingly.

[0758] A "recording device" is a device used to record anomaly detection data and manage its history.

[0759] A "feedback device" is a device that has the function of adjusting the generated AI model based on feedback information in order to improve the accuracy of anomaly detection.

[0760] A "generative AI model" is an artificial intelligence model that can be adjusted to improve the accuracy of anomaly detection based on feedback.

[0761] To implement this invention, a camera is first required to identify the target objects, such as machinery and robots within a factory, and to monitor their operation in real time. This camera is installed as an AI camera and covers the operating range of the target objects. In addition, a server is required to process the captured video data and detect anomalies; this server acts as a processing unit.

[0762] The server uses OpenCV to analyze video data and analyzes behavioral patterns in real time. Based on these behavioral patterns, a generative AI model using TensorFlow detects anomalies and immediately notifies the administrator. Furthermore, the server uses an NLP library to analyze the administrator's emotional state and flexibly adjusts the notification content when the user is experiencing stress.

[0763] Furthermore, this system utilizes a recording device to record anomaly detection data, and a feedback device adjusts the generated AI model based on past anomaly detection records, thereby improving the accuracy of anomaly detection.

[0764] As a concrete example, the system detects abnormal vibrations occurring on a conveyor belt in a factory and uses this information to notify the manager of a potential overload condition. If the manager is under stress, the notification can be made in a gentler tone, such as, "There are some slight vibrations on the conveyor belt. No immediate action is required, but please check it periodically."

[0765] An example of a prompt message would be: "Use the AI ​​camera to monitor the robot's vibration patterns, and if an anomaly is detected, adjust the notification method according to the administrator's emotional state. Emotion analysis will help select the most appropriate notification format."

[0766] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0767] Step 1:

[0768] The AI ​​camera captures the movements of the target object, such as a machine or robot.

[0769] The input is real-time video of the object, and the output is recorded video data. The camera continuously acquires video and transmits the data to the server in real time.

[0770] Step 2:

[0771] The server uses OpenCV to analyze the video data and extract the motion patterns of the target object.

[0772] The input is the video data obtained in step 1, and the output is the motion pattern data. The server uses computer vision technology to extract motion features and convert the data into a format usable in the next processing step.

[0773] Step 3:

[0774] The server uses TensorFlow to perform anomaly detection using a generative AI model.

[0775] The input is the operation pattern data obtained in step 2, and the output is the anomaly detection result data. In this step, anomalies are identified by comparing them with known normal patterns. If an anomaly is detected, the information is sent to the next notification step.

[0776] Step 4:

[0777] The server uses an NLP library to analyze the administrator's emotional state.

[0778] The input is voice and text data from the administrator, and the output is emotional state data. Based on this emotional data, the server assesses the administrator's current stress level and prepares to generate appropriate notification content accordingly.

[0779] Step 5:

[0780] The server generates notification content tailored to the anomaly and the administrator's emotional state, and sends it to the administrator's terminal.

[0781] The input consists of the anomaly detection result data from step 3 and the emotional state data from step 4, and the output is a notification message to the administrator. This notification is flexibly adjusted according to the administrator's status and sent immediately.

[0782] Step 6:

[0783] The server saves the anomaly detection data to a recording device, and the AI ​​model is readjusted by a feedback device.

[0784] The input is anomaly detection result data, and the output is an updated generative AI model. In this step, the history is saved and used as feedback to train the AI ​​model, thereby improving the model's accuracy.

[0785] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0786] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0787] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0788] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0789] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0790] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0791] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0792] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0793] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0794] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0795] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0796] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0797] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0798] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0799] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0800] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0801] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0802] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0803] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0804] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0805] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0806] The following is further disclosed regarding the embodiments described above.

[0807] (Claim 1)

[0808] This is a video acquisition device for filming animal behavior, and includes means for installation within the animal's habitat.

[0809] A processing device for analyzing captured video data in real time and analyzing animal behavior patterns,

[0810] A notification device means that immediately notifies of an abnormality detected by the aforementioned analysis device,

[0811] A recording device means for recording anomaly detection data and managing its history,

[0812] A feedback device means for adjusting the AI ​​model based on feedback in order to improve the accuracy of anomaly detection,

[0813] A system that includes this.

[0814] (Claim 2)

[0815] The processing device according to claim 1, comprising an algorithm for distinguishing between normal behavioral patterns and abnormal behavior based on the movement speed, trajectory, and posture of an animal.

[0816] (Claim 3)

[0817] The system according to claim 1, further comprising a suggestion device for notifying the user of appropriate countermeasures based on the results of animal behavior analysis.

[0818] "Example 1"

[0819] (Claim 1)

[0820] This is an image acquisition method for photographing animal behavior, and is installed in the living space of the organism.

[0821] An information processing means for analyzing the behavioral patterns of organisms by analyzing captured image data in real time,

[0822] A communication means for immediately notifying of an anomaly detected by the information processing means,

[0823] A recording means for recording detected abnormal data and managing its history,

[0824] Correction means for adjusting the learning model based on feedback in order to improve the accuracy of anomaly detection,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The information processing means according to claim 1, comprising a calculation formula for distinguishing between normal behavior and abnormal behavior based on the movement speed, path, and positional changes of an organism.

[0828] (Claim 3)

[0829] The system according to claim 1, further comprising a means for notifying the user of appropriate countermeasures based on the results of behavioral analysis of organisms.

[0830] "Application Example 1"

[0831] (Claim 1)

[0832] This is a video acquisition device for capturing the behavior of a moving object, and includes means installed at the work area of ​​the moving object.

[0833] A processing device for analyzing captured video data in real time and analyzing the behavioral patterns of moving objects,

[0834] A notification device means that immediately notifies of an abnormality detected by the aforementioned analysis device,

[0835] A recording device means for recording anomaly detection data and managing its history,

[0836] A feedback device means for adjusting a machine learning model based on feedback in order to improve the accuracy of anomaly detection,

[0837] A system that includes this.

[0838] (Claim 2)

[0839] The processing apparatus according to claim 1, comprising an algorithm for distinguishing between normal behavior patterns and abnormal behavior based on the movement speed, trajectory, and attitude of a moving object.

[0840] (Claim 3)

[0841] The system according to claim 1, further comprising a presentation device for notifying the user of appropriate countermeasures based on the results of behavioral analysis of an operating object.

[0842] "Example 2 of combining an emotion engine"

[0843] (Claim 1)

[0844] An imaging device for photographing animal behavior, and a means for installing it in the area of ​​living organisms,

[0845] An analytical device means for immediately analyzing captured visual data and analyzing the behavioral characteristics of organisms,

[0846] A notification device means for quickly notifying abnormal information detected by the aforementioned analysis device,

[0847] A storage device means for storing and maintaining records of anomaly detection data,

[0848] An adjustment device means for adjusting a machine learning model based on a response in order to improve the accuracy of anomaly detection,

[0849] An emotion analysis device means for analyzing the user's emotional information and adjusting the communication method,

[0850] A system that includes this.

[0851] (Claim 2)

[0852] The analysis device according to claim 1, comprising a method for distinguishing between normal behavioral characteristics and abnormal behavior based on the movement speed, path, and posture of an organism.

[0853] (Claim 3)

[0854] The system according to claim 1, further comprising a suggestion device for recommending appropriate measures based on the user's emotional state.

[0855] "Application example 2 when combining with an emotional engine"

[0856] (Claim 1)

[0857] This is an image acquisition device for capturing the movement of an object, and includes means installed within the movement range of the object.

[0858] A processing device for analyzing captured video data in real time and analyzing the motion patterns of an object,

[0859] A notification device means that immediately notifies of an abnormality detected by the aforementioned analysis device,

[0860] An emotion analysis device means for analyzing the user's emotional state and adjusting the notification content,

[0861] A recording device means for recording anomaly detection data and managing its history,

[0862] A feedback device means for adjusting the generated AI model based on feedback in order to improve the accuracy of anomaly detection,

[0863] A system that includes this.

[0864] (Claim 2)

[0865] The processing apparatus according to claim 1, comprising an algorithm for distinguishing between normal operation patterns and abnormal operation based on the movement speed, trajectory, and posture of an object.

[0866] (Claim 3)

[0867] The system according to claim 1, further comprising a suggestion device for notifying the user of appropriate countermeasures based on the results of motion analysis of an object. [Explanation of Symbols]

[0868] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. This is a video acquisition device for filming animal behavior, and includes means for installation within the animal's habitat. A processing device for analyzing captured video data in real time and analyzing animal behavior patterns, A notification device means that immediately notifies of an abnormality detected by the aforementioned analysis device, A recording device means for recording anomaly detection data and managing its history, A feedback device means for adjusting the AI ​​model based on feedback in order to improve the accuracy of anomaly detection, A system that includes this.

2. The processing device according to claim 1, comprising an algorithm for distinguishing between normal behavioral patterns and abnormal behavior based on the animal's movement speed, trajectory, and posture.

3. The system according to claim 1, further comprising a suggestion device for notifying the user of appropriate countermeasures based on the results of animal behavior analysis.

Images