system

Abstract

Provide a system. 【Solution means】 Means for acquiring position information, Means for analyzing the acquired position information, Means for modeling the normal behavior range based on the analysis, Means for detecting an abnormality by comparison with the normal behavior range, Means for generating an alarm when an abnormality is detected, Means for notifying the user of the alarm, Means for communicating with an external organization based on the alarm, Means for monitoring a moving object in a home environment, Means for controlling an image recording device when an abnormality is detected, Means for alerting with a voice output means when an abnormality occurs, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is an important issue for the whole society to ensure the safety of children. In particular, due to unexpected actions and movements, the risk of children getting lost cannot be ignored. Currently, there is no efficient method to immediately grasp the situation when a child deviates from the normal movement range. In addition, means for detecting abnormalities and taking appropriate actions promptly are required.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides a means for acquiring location information and modeling the normal range of movement. This enables the detection of anomalies in real time when the user deviates from the normal range of movement and promptly notifies the user of an alert. Furthermore, in the event of an anomaly, the invention provides a means for analyzing the behavioral pattern using a machine learning algorithm and for coordinating with appropriate external organizations, thereby enabling a rapid and effective response.

[0006] "Means of acquiring location information" refers to a function for collecting GPS data from wearable devices worn by children.

[0007] "Means for analyzing location information" refers to processing functions that evaluate a child's movement patterns based on acquired location data.

[0008] "Methods for modeling normal range of activity" refer to techniques that involve registering and updating a database of a child's movement paths and activity areas in a steady state, and then constructing a model based on that data.

[0009] "Means for detecting anomalies" are algorithms that identify when a monitored entity takes unexpected actions by comparing them to a normal behavior model.

[0010] "Means of generating warnings" refers to a function that sends a warning signal to parents or relevant authorities when an anomaly occurs.

[0011] "Means of communication with external organizations" refer to communication protocols and equipment that enable rapid response by contacting the police and other rescue organizations as needed. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the language used in the following description will be explained. [[ID=四十八]]

[0015] [[ID=四十九]] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the labeled communication I / F (Interface) is an interface that includes a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is a system designed to ensure the safety of children, enabling rapid response by acquiring location information through a wearable device and detecting abnormal behavior. The device operates as follows:

[0034] Device-level operation

[0035] The device functions as a wearable device worn by children. It has a GPS sensor for individually acquiring location information and can also analyze detailed movement patterns using an accelerometer. The collected data is sent to a server in fixed batch sizes. The device has pre-set areas for the child's usual school route and activity range, and records and analyzes their movements within those areas.

[0036] Data analysis on the server

[0037] The server analyzes location and movement data received from the terminal. This allows it to model the normal behavioral patterns of each child, and an algorithm operates to detect abnormal movements in real time based on these criteria. Machine learning techniques are used in this analysis, and the model is continuously improved based on new data.

[0038] Anomaly detection and notification

[0039] When an anomaly is detected, the server immediately generates an alert and sends a notification to the user's (parent's) smartphone. This notification includes the location and time of the anomaly, as well as the current location information. Based on this, parents can take immediate action.

[0040] Collaboration with external organizations

[0041] If a parent or guardian notices an abnormal situation and determines that immediate action is necessary, the device has communication capabilities to connect with pre-configured external organizations, such as the police or local crime prevention groups. This allows for the transmission of the child's precise location and situation information to the relevant organizations, enabling prompt intervention.

[0042] For example, if a child deviates from their usual route home from school and stays in an unfamiliar location, the server detects the anomaly and immediately sends an alert to the parent. This alert allows the parent to call their child to check on their safety or, if necessary, notify the police. Through these processes, a comprehensive solution is provided to protect children's safety.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The device uses its built-in GPS sensor to acquire the child's location information at regular intervals. This information is temporarily stored in the device's memory.

[0046] Step 2:

[0047] The device also collects data from its accelerometer and performs a basic check on the spot to analyze its movement pattern and detect any abnormalities. It then prepares to send the results, along with location information, to the server in batches.

[0048] Step 3:

[0049] After receiving location and movement data transmitted from the terminal, the server models the typical range of movement by comparing it with historical data.

[0050] Step 4:

[0051] The server compares the latest data with a normal behavior model and sets an anomaly detection flag if any behavior deviating from the standard is detected.

[0052] Step 5:

[0053] If an anomaly detection flag is set, the server immediately generates an alert and pushes the details to the user (parent / guardian)'s smartphone application.

[0054] Step 6:

[0055] Users can review received alerts and view their child's current location and movement history from the application interface.

[0056] Step 7:

[0057] If the user determines that the abnormality will persist, they can instruct the application to contact external organizations such as the police.

[0058] Step 8:

[0059] After being instructed to contact an external organization, the device continuously transmits location information to the server and performs the procedure of sending necessary information to the external organization while maintaining the latest information.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In recent years, there has been a growing demand for ensuring the safety of children, and monitoring their location during school commutes and daily activities, as well as the early detection of abnormal behavior, have become crucial issues. However, conventional systems lacked sufficient real-time dynamic data analysis, sometimes resulting in delays in detecting anomalies. Furthermore, the rapid notification of detected anomalies and inadequate coordination with external organizations failed to alleviate the anxieties of parents and guardians.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for collecting location data, means for analyzing the collected location data, and means for generating a general range of movement based on the analysis. This makes it possible to monitor a child's location in real time, send notifications quickly when an anomaly is detected, and cooperate with external organizations as needed.

[0065] "Location data" refers to latitude and longitude information used to identify a physical location. It may also include information about time and speed.

[0066] "Analysis" refers to the process of analyzing collected data using statistical or algorithmic methods to identify meaningful patterns or anomalies.

[0067] "General range of activity" is a model that represents the normal range of activity for a particular individual or object, and is created based on past data.

[0068] "An anomaly" refers to a phenomenon that deviates from a predetermined normal range of behavior or behavioral patterns.

[0069] "Notification" refers to a system message that provides alerts or warnings to users or relevant organizations when an anomaly is detected.

[0070] "Information exchange with other organizations" refers to the procedure of coordinating with external organizations and groups as needed to share location information and detailed information about the situation when an anomaly is detected.

[0071] This invention is a location data monitoring system for ensuring the safety of children. The system consists of a wearable device, a server for data analysis, and a user's smart device for receiving alerts.

[0072] The device functions as a small wearable device worn by children, collecting location and movement data using a GPS sensor and an accelerometer. Specifically, a chip embedded in the device collects latitude and longitude information, as well as movement speed and acceleration, at regular intervals, and sends this data to a server in batch format. HTTPS is used as the communication protocol for data transmission to ensure data security.

[0073] The server analyzes the data received from the terminal using machine learning algorithms. Preferably, the analysis is performed using the Python library scikit-learn to model normal behavioral patterns. Based on the model, the server evaluates the current data and immediately generates an alert if an anomaly is detected. This alert is sent to the user's smart device via the Firebase Cloud Messaging service.

[0074] Users receive alerts via their smartphones. These alerts include the specific time and location where an anomaly was detected, enabling a rapid response. Through the application's features, users can, if necessary, send information to external organizations such as the police or local crime prevention groups to request a swift response.

[0075] For example, if a child takes an unusual route home from school, the server will detect the anomaly and immediately notify the parent. The parent can then check on the child's safety and, if necessary, contact the police.

[0076] Example prompt for a generated AI model: "Design an anomaly detection model that uses a child's location data to detect deviations from their normal behavior patterns and notify parents. This model aims to monitor the route home from school and detect unnatural route changes in real time."

[0077] This system makes it easier for parents to ensure their children's safety and take necessary measures.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The device functions as a wearable device worn by children and periodically acquires location and movement data (e.g., every 5 seconds) using GPS and accelerometer sensors. Inputs include the object's latitude and longitude, as well as its speed and direction of movement, while output is in raw data format. This data is temporarily stored within the device and prepared for transmission in batches at regular intervals.

[0081] Step 2:

[0082] The terminal sends the collected location data in batch format to the server. The input is the location and movement data collected in step 1, and the output is batched data packets. Secure and safe communication is ensured by using the HTTPS protocol for data transmission.

[0083] Step 3:

[0084] The server receives location data transmitted from the terminal and stores it in a database. The input is the raw location data transmitted from the terminal, and the output is the stored data. Simultaneously with saving, the server prepares to begin data analysis based on the received data.

[0085] Step 4:

[0086] The server runs machine learning algorithms to analyze the stored data. This analysis uses the scikit-learn library in Python to model typical behavioral patterns based on historical data. The input is historical location and movement data, and the output is a typical behavioral pattern model. This model serves as a benchmark for evaluating new data.

[0087] Step 5:

[0088] The server uses the model obtained from the analysis to evaluate how much the newly received data deviates from the normal range of movement. The input is the latest location data and behavior pattern model sent in step 2, and the output is a determination of whether or not an anomaly is present. If an anomaly is detected, that information is used in the next step.

[0089] Step 6:

[0090] The server immediately generates an alert upon detecting an anomaly. This alert includes information about the specific time and location where the anomaly was detected. The input is whether or not an anomaly was detected, as determined in step 5, and the output is the alert information. After the alert is generated, the notification system starts operating.

[0091] Step 7:

[0092] The server sends the generated alerts to the user's smart device via Firebase Cloud Messaging. The input is the generated alert information, and the output is a notification to the user. This notification allows the user to immediately understand their child's unusual behavior and consider necessary actions.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] Monitoring children's safety at home is a major challenge for parents. Especially in recent years, with the increase in dual-income households and heightened awareness of crime prevention, there is a growing need to understand any unusual situations within the home in real time and respond quickly. However, conventional safety monitoring systems lack sufficient coverage and functionality, making effective responses difficult.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for acquiring location information, means for controlling an image recording device when an anomaly is detected, and means for issuing a warning via voice output when an anomaly occurs. This enables monitoring of children's safety within the home in a variety of ways and allows for a rapid response.

[0098] "Means for acquiring location information" refers to technologies and devices for determining the current location of a moving object.

[0099] "Means for analyzing acquired location information" refers to functions and algorithms for evaluating behavioral patterns and anomalies based on the obtained location data.

[0100] "Means for modeling the normal range of movement" refers to a function that uses mathematical or statistical methods to represent the normal behavior and range of movement of a moving object obtained from repeated actions.

[0101] A "means for detecting anomalies" is a system that identifies discrepancies between a modeled normal range of movement and actual movements, and signals an abnormal situation.

[0102] "Means of generating alarms" refers to the process of generating signals or messages to quickly notify users when an anomaly is detected.

[0103] "Means of notifying users" refers to communication means or devices used to transmit alarms or information to users.

[0104] "Means of communication with external organizations" refers to systems and technologies for transmitting information to predetermined external organizations or individuals.

[0105] "Means for monitoring moving objects in a home environment" refers to a system for continuously observing and recording the movements of moving objects within a home.

[0106] "Means for controlling an image recording device" refers to operations or methods for managing the operation of a device that records images or videos under specific circumstances.

[0107] "Means for issuing warnings via voice output in the event of an abnormality" refers to methods for conveying warnings or instructions using voice when an abnormality occurs.

[0108] The system for implementing this invention operates using several key components to monitor the safety of children in a home environment. The server acquires location information and detects anomalies by comparing it with a modeled normal range of activity. If an anomaly is detected, it generates an alarm and promptly notifies the user. Furthermore, it has the function of controlling mobile monitoring and image recording devices to monitor the situation within the home. In the event of an anomaly, the user can be alerted using an audio output means.

[0109] The specific system configuration involves a wearable device measuring the child's location in real time and transmitting this information to a server. The server analyzes the received location data and generates normal behavioral patterns using machine learning algorithms such as TENSORFLOW® and PyTorch. When an anomaly is detected, the server controls image recording devices such as cameras and video recorders, and simultaneously issues necessary warnings using an audio output device.

[0110] For example, if a child deviates from their usual range of movement, the server immediately detects the anomaly and begins recording images. It can also issue audio alerts and provide appropriate instructions to protect the child. This allows parents to immediately understand the situation via their smartphones and, if necessary, coordinate with external organizations.

[0111] A concrete example of using a generative AI model is the ability to generate a prompt message such as, "Develop a system that ensures children are playing safely at home and notifies the user if they go outside a defined area." In this way, the system can provide a safe home environment and offer overall support to ensure children's safety.

[0112] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0113] Step 1:

[0114] The device acquires the child's location information through a wearable device. It receives GPS information and acceleration data as input, formats them into a data format for transmission to the server, and sends the formatted data to the server as output.

[0115] Step 2:

[0116] The server analyzes location and acceleration data received from the terminal. It receives data from the terminal as input and analyzes normal behavior patterns using machine learning algorithms (e.g., TensorFlow). During the data processing process, it performs noise reduction and data smoothing, and generates normal behavior patterns as output.

[0117] Step 3:

[0118] The server analyzes newly received location information based on the generated normal behavior pattern and detects whether there is an anomaly. It takes new location information as input and compares it with the previous pattern. If an anomaly pattern is found, it sets an anomaly detection flag as output.

[0119] Step 4:

[0120] If an anomaly is detected, the server generates an alarm and notifies the user's smartphone of this information. It receives an anomaly detection flag as input and generates the notification content. The output is sent to the user in the form of a text message or push notification.

[0121] Step 5:

[0122] The server simultaneously controls the video recording device installed in the home and issues a command to record the situation when an anomaly is detected. It uses an anomaly detection flag as input to generate the recording command. The output is a control signal to the recording device.

[0123] Step 6:

[0124] The server operates an audio output device in the event of an anomaly, sending an audio message to alert the child. It uses alarm information as input to generate the audio message. The output is an audio signal sent to the audio device.

[0125] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0126] This invention is a system for protecting children's safety, and by taking user emotions into consideration, it provides more flexible and effective warnings and responses. This system operates primarily by utilizing wearable devices, servers, smartphone applications, and an emotion engine.

[0127] Operation in wearable devices

[0128] The device is designed as a wearable device for children and is equipped with a GPS sensor and an accelerometer. It collects the child's location information and movement patterns in real time and transmits them to a server. If the child deviates from their normal range of activity, this information is immediately recorded as abnormal data.

[0129] Server-based data analysis and the use of emotion engines

[0130] The server has the capability to analyze data received from the terminal and detect anomalies by comparing them to the user's normal range of activity. Furthermore, an emotion engine analyzes the user's current emotional state. Based on the results of this emotion analysis, the urgency and content of the alert are adjusted. For example, if the user is feeling excessively anxious, the notification may be changed to calmer language or support information may be added.

[0131] Generation of alarm notifications and coordination with external organizations

[0132] If abnormal behavior is detected, the server generates an alarm and sends it to the user. The way the alarm should be expressed is determined based on emotional information provided by the emotion engine. The user can also check the child's location and movement history through a smartphone app and decide whether to contact an external organization. If contact with an external organization is necessary, the communication will be made at an appropriate time and with appropriate content, based on the analysis results of the emotion engine.

[0133] Specific example

[0134] For example, if a child deviates from their usual route home and stops at a certain location for an extended period, the server detects this as an anomaly and notifies the parent. If the emotion engine analyzes the situation and determines that the parent is highly anxious, the notification will include a reassuring message. Furthermore, if the user chooses to contact the police, prompt and appropriate action will be required, taking into account the emotional information.

[0135] Thus, the present invention provides a system that can effectively protect children's safety by integrating data analysis, sentiment analysis, and rapid collaboration.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The device uses a built-in GPS sensor to collect the child's current location at regular intervals. It also acquires movement data using an accelerometer and sends all of this data together to a server.

[0139] Step 2:

[0140] The server analyzes location and movement data received from the terminal and evaluates the current situation based on a normal behavior model. If abnormal movement is detected, a flag is set.

[0141] Step 3:

[0142] The server prompts the emotion engine to evaluate the user's current emotions, regardless of whether an anomaly flag is present. The emotion engine infers and analyzes emotions based on audio data, touch data, and other information obtained from the user's smartphone or wearable device.

[0143] Step 4:

[0144] The server adjusts the content of the alarms it generates when an anomaly is detected, based on the output of the emotion engine. If it determines that the user is experiencing anxiety, it makes adjustments such as softening the notification message.

[0145] Step 5:

[0146] Users check the alerts they receive on their smartphones, pinpoint their child's current location based on the information provided, and view their activity history within the app to understand the situation.

[0147] Step 6:

[0148] Users can contact external organizations such as the police through the app as needed. If the emotion engine determines that the user is in an anxious state, a guide will be added to the notification to help them contact external organizations more smoothly.

[0149] Step 7:

[0150] The device confirms that contact has been made with an external organization, continuously updates its location information in real time, and provides the latest information to the external organization via the server.

[0151] (Example 2)

[0152] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0153] Simply detecting anomalies and issuing alarms to ensure children's safety can cause excessive anxiety among parents. In such situations, cold, emotionless notifications can actually increase anxiety and hinder a swift and appropriate response. Therefore, alarm notifications that take the user's emotional state into consideration are necessary.

[0154] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0155] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, means for modeling the normal range of activity based on the analysis, means for detecting anomalies by comparing with the normal range of activity, means for generating an alarm when an anomaly is detected, means for notifying the user of the alarm, means for communicating with an external organization based on the alarm, means for analyzing the user's emotional state and adjusting the content of the alarm, and means for changing the urgency of the alarm based on emotional information when an anomaly is detected. This enables flexible and effective alarm notifications that respond to the user's emotional state, and is expected to allow for a quick and appropriate response to ensure the safety of children.

[0156] "Location information" refers to data that indicates the geographical location of a specific object or individual.

[0157] "Analysis" is the process of examining collected information and data in detail and extracting patterns and features.

[0158] "Normal range of movement" refers to the area and route that the subject is expected to normally travel, modeled based on past behavioral data.

[0159] "Anomaly detection" is the process of identifying movements that deviate from the normal, expected range of activity or behavioral patterns, and determining that these movements are abnormal.

[0160] An "alert" is a signal or message used to notify a user that attention or caution is necessary when specific conditions or situations occur.

[0161] An "external organization" refers to an organization or group that exists outside the system and with which communication and collaboration are possible.

[0162] "Emotional state" refers to the psychological state or emotional changes an individual is experiencing at a given time.

[0163] "Urgency" is a measure that indicates how quickly a response is required to a particular situation or event.

[0164] This invention is a system designed to ensure the safety of children, utilizing a wearable device, a server, a smartphone application, and an emotion engine. The invention proceeds roughly as follows:

[0165] Use of wearable devices

[0166] The device functions as a wearable device for children. It is equipped with a GPS sensor for acquiring location information and an accelerometer for detecting abnormal movement. The device collects location information and movement patterns in real time and transmits this data to a server. This data is then used to prepare for the detection of deviations from the child's normal range of activity.

[0167] Server Role

[0168] The server analyzes location information based on received data and models the user's normal range of movement. Furthermore, the server detects anomalies by comparing the current location to the normal range of movement. If an anomaly is detected, it generates an alarm and sends it to the user as a notification. In addition, the server is equipped with an emotion engine that analyzes the user's emotional state. Based on the results of this analysis, the urgency and content of the alarm message are adjusted.

[0169] User interaction

[0170] Users can receive alert notifications through a smartphone app and check their child's location and movement history in real time. If necessary, users can contact external organizations, and the system will assist in making such contacts at the optimal time and with the most appropriate content, based on emotional information.

[0171] As a concrete example, a user can input a prompt message to the AI ​​model saying, "When abnormal behavior in a child is detected, please generate a message that will reassure the parent," which will help generate emotionally sensitive messages.

[0172] This system ensures that each function works effectively together, enabling a quick and appropriate response to protect children's safety.

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] The device acquires location information and movement patterns via a wearable device worn by the child. Specifically, it measures location in real time using a GPS sensor and captures changes in movement using an accelerometer. This data is initially stored locally and then sent to a server. The input is sensor data from the wearable device, and the output is data transmission to the server.

[0176] Step 2:

[0177] The server receives data transmitted from the terminal. It analyzes location and acceleration data and compares it to past patterns to model the normal range of movement. This analysis prepares the server to detect deviations from normal behavior. The input is the data transmitted by the terminal, and the output is the result of the behavior pattern analysis. Based on these results, the server prepares the data for the next step.

[0178] Step 3:

[0179] The server detects anomalies by comparing the normal range of activity with the latest data. When an anomaly is detected, the server flags it as an anomaly and generates an alarm. Specifically, it determines the content and urgency of the alarm based on the type and severity of the abnormal behavior. The input is the result of the behavior pattern analysis, and the output is an instruction to generate an alarm.

[0180] Step 4:

[0181] The server analyzes the user's emotional state using an emotion engine. For example, it identifies anxiety and reassurance by considering alarms and conditioned responses the user has received in the past. This analysis result is used to determine the content and tone of the next alarm message to be generated. The input is interaction data with the user, and the output is the result of the emotional state analysis.

[0182] Step 5:

[0183] The server generates an appropriate alarm message based on the results of anomaly detection and sentiment analysis, and notifies the user. This message incorporates information from the sentiment engine. Upon receiving this notification, the user checks the details via a smartphone app and takes immediate action as needed. The input is the server's analysis result, and the output is the final alarm message sent to the user.

[0184] Step 6:

[0185] Users receive alert notifications through a smartphone app and can check their child's location and movement history. If a user determines that contacting an external agency is necessary, they can use the support prompts built into the app to make contact at the optimal time and with appropriate content, taking emotional information into consideration. The input is an alert message from the server, and the output is the user's action of contacting an external agency.

[0186] (Application Example 2)

[0187] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0188] Ensuring child safety is a critical issue in modern society. However, conventional location-based systems fail to consider the emotional state of parents when detecting anomalies and issuing alarms, potentially causing excessive anxiety. Therefore, there is a need for technology that enables flexible and effective responses that take parents' emotions into account when detecting anomalies and generating alarms based on location information.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0190] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, and means for adjusting the content of the alarm based on the analyzed emotional state. This makes it possible to issue an appropriate alarm that provides reassurance, taking into account the emotional state of the guardian, when an abnormality in the child's safety is detected.

[0191] "Location information" refers to data that indicates the specific geographical location of an object.

[0192] "Means of analysis" refers to methods or devices for analyzing collected data and extracting useful information from that data.

[0193] "Normal range of activity" is a model of the geographical area that a subject typically travels within on a daily basis.

[0194] "Means for detecting anomalies" refers to methods or devices for identifying movements that deviate from normal behavioral patterns.

[0195] "Means for generating an alarm" refers to a method or device for creating an alert to draw attention when an anomaly is detected.

[0196] "User emotional state" refers to the psychological or emotional situation a user is currently experiencing.

[0197] "Means for adjusting the content of an alarm based on emotional state" refers to a method or apparatus for optimizing an alarm message in accordance with the user's emotional analysis.

[0198] "Means of notification" refers to a method or device for transmitting information or a message to a designated recipient.

[0199] "Means of communication with external organizations" refers to methods or devices that a system uses to exchange information with external organizations or services.

[0200] In the system implementing this invention, a wearable device first acquires the child's location information in real time and transmits it to a server. The wearable device is equipped with a GPS sensor and an accelerometer. The server analyzes the received location information and detects anomalies by comparing it with the child's normal range of activity. At this time, the server uses a machine learning algorithm to model the child's normal range of activity based on past behavioral patterns.

[0201] When an anomaly is detected, the server generates an alarm and notifies the user. Furthermore, it uses an emotion engine to analyze the user's emotional state. The emotion engine utilizes a generative AI model to analyze the user's emotions and provides a means to adjust the content of the alarm based on the results. For example, if the user is feeling anxious, the notification will include a reassuring message.

[0202] Users can check their child's current location and movement history through a smartphone application. Based on this information, they can determine whether it is necessary to contact an external agency. When communicating with an external agency, the server will send a notification based on emotional information at the appropriate time.

[0203] For example, if a child deviates from their usual route to school and is stopped for an extended period at an unexpected location, the server will consider this information suspicious and send a notification to the parent. The AI ​​model then adds considerate wording to the notification for anxious parents, such as, "Your child has been confirmed safe. Please remain calm and check the situation."

[0204] Examples of prompt messages include, "Generate a calm and reassuring notification message based on the user's emotions," and "Create a reassuring notification message for parents if abnormal behavior is detected."

[0205] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0206] Step 1:

[0207] The device uses GPS and accelerometer sensors to acquire the child's location and movement data in real time. This data, including the child's current location and movement patterns, is transmitted from the device to the server. The input is location and velocity data from the sensors, and the output is raw data sent to the server.

[0208] Step 2:

[0209] The server analyzes the received location information and models the normal range of movement by comparing it with past location information. This process uses machine learning algorithms to generate normal behavior patterns and establish criteria for detecting anomalies. The input is location information from the terminal, and the output is a model of the normal range of movement.

[0210] Step 3:

[0211] The server detects an anomaly when location information deviates from the normal range of activity. Upon detection of an anomaly, it generates an alarm and proceeds to the next step. In this step, real-time location data is evaluated based on the generated normal range model to identify the anomaly. The input is real-time location information, and the output is an anomaly detection flag.

[0212] Step 4:

[0213] The server uses a generative AI model to analyze the user's emotions. In this analysis, it detects the user's current emotional state and adjusts the alarm content accordingly. The input is emotional data obtained from the user (e.g., questionnaires for emotion analysis or facial recognition data), and the output is the user's emotional state.

[0214] Step 5:

[0215] If an anomaly is detected, the server adjusts the alarm content based on the user's emotional state and sends a notification to the user's smartphone. This notification includes an emotionally sensitive message, aiming to provide a sense of reassurance. The input is the anomaly detection flag and the adjusted alarm content, and the output is the notification message to the user.

[0216] Step 6:

[0217] Users receive notifications and can use a smartphone application to check their child's location and movement history in real time. They can also consider contacting external organizations if necessary. Input is notification messages from the server, and output is a log of the user's decision-making.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] This invention is a system designed to ensure the safety of children, enabling rapid response by acquiring location information through a wearable device and detecting abnormal behavior. The device operates as follows:

[0235] Device-level operation

[0236] The device functions as a wearable device worn by children. It has a GPS sensor for individually acquiring location information and can also analyze detailed movement patterns using an accelerometer. The collected data is sent to a server in fixed batch sizes. The device has pre-set areas for the child's usual school route and activity range, and records and analyzes their movements within those areas.

[0237] Data analysis on the server

[0238] The server analyzes location and movement data received from the terminal. This allows it to model the normal behavioral patterns of each child, and an algorithm operates to detect abnormal movements in real time based on these criteria. Machine learning techniques are used in this analysis, and the model is continuously improved based on new data.

[0239] Anomaly detection and notification

[0240] When an anomaly is detected, the server immediately generates an alert and sends a notification to the user's (parent's) smartphone. This notification includes the location and time of the anomaly, as well as the current location information. Based on this, parents can take immediate action.

[0241] Collaboration with external organizations

[0242] If a parent or guardian notices an abnormal situation and determines that immediate action is necessary, the device has communication capabilities to connect with pre-configured external organizations, such as the police or local crime prevention groups. This allows for the transmission of the child's precise location and situation information to the relevant organizations, enabling prompt intervention.

[0243] For example, if a child deviates from their usual route home from school and stays in an unfamiliar location, the server detects the anomaly and immediately sends an alert to the parent. This alert allows the parent to call their child to check on their safety or, if necessary, notify the police. Through these processes, a comprehensive solution is provided to protect children's safety.

[0244] The following describes the processing flow.

[0245] Step 1:

[0246] The device uses its built-in GPS sensor to acquire the child's location information at regular intervals. This information is temporarily stored in the device's memory.

[0247] Step 2:

[0248] The device also collects data from its accelerometer and performs a basic check on the spot to analyze its movement pattern and detect any abnormalities. It then prepares to send the results, along with location information, to the server in batches.

[0249] Step 3:

[0250] After receiving location and movement data transmitted from the terminal, the server models the typical range of movement by comparing it with historical data.

[0251] Step 4:

[0252] The server compares the latest data with a normal behavior model and sets an anomaly detection flag if any behavior deviating from the standard is detected.

[0253] Step 5:

[0254] If an anomaly detection flag is set, the server immediately generates an alert and pushes the details to the user (parent / guardian)'s smartphone application.

[0255] Step 6:

[0256] Users can review received alerts and view their child's current location and movement history from the application interface.

[0257] Step 7:

[0258] If the user determines that the abnormality will persist, they can instruct the application to contact external organizations such as the police.

[0259] Step 8:

[0260] After being instructed to contact an external organization, the device continuously transmits location information to the server and performs the procedure of sending necessary information to the external organization while maintaining the latest information.

[0261] (Example 1)

[0262] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0263] In recent years, there has been a growing demand for ensuring the safety of children, and monitoring their location during school commutes and daily activities, as well as the early detection of abnormal behavior, have become crucial issues. However, conventional systems lacked sufficient real-time dynamic data analysis, sometimes resulting in delays in detecting anomalies. Furthermore, the rapid notification of detected anomalies and inadequate coordination with external organizations failed to alleviate the anxieties of parents and guardians.

[0264] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0265] In this invention, the server includes means for collecting location data, means for analyzing the collected location data, and means for generating a general range of movement based on the analysis. This makes it possible to monitor a child's location in real time, send notifications quickly when an anomaly is detected, and cooperate with external organizations as needed.

[0266] "Location data" refers to latitude and longitude information used to identify a physical location. It may also include information about time and speed.

[0267] "Analysis" refers to the process of analyzing collected data using statistical or algorithmic methods to identify meaningful patterns or anomalies.

[0268] "General range of activity" is a model that represents the normal range of activity for a particular individual or object, and is created based on past data.

[0269] "An anomaly" refers to a phenomenon that deviates from a predetermined normal range of behavior or behavioral patterns.

[0270] "Notification" refers to a system message that provides alerts or warnings to users or relevant organizations when an anomaly is detected.

[0271] "Information exchange with other organizations" refers to the procedure of coordinating with external organizations and groups as needed to share location information and detailed information about the situation when an anomaly is detected.

[0272] This invention is a location data monitoring system for ensuring the safety of children. The system consists of a wearable device, a server for data analysis, and a user's smart device for receiving alerts.

[0273] The device functions as a small wearable device worn by children, collecting location and movement data using a GPS sensor and an accelerometer. Specifically, a chip embedded in the device collects latitude and longitude information, as well as movement speed and acceleration, at regular intervals, and sends this data to a server in batch format. HTTPS is used as the communication protocol for data transmission to ensure data security.

[0274] The server analyzes the data received from the terminal using machine learning algorithms. Preferably, the analysis is performed using the Python library scikit-learn to model normal behavioral patterns. Based on the model, the server evaluates the current data and immediately generates an alert if an anomaly is detected. This alert is sent to the user's smart device via the Firebase Cloud Messaging service.

[0275] Users receive alerts via their smartphones. These alerts include the specific time and location where an anomaly was detected, enabling a rapid response. Through the application's features, users can, if necessary, send information to external organizations such as the police or local crime prevention groups to request a swift response.

[0276] For example, if a child takes an unusual route home from school, the server will detect the anomaly and immediately notify the parent. The parent can then check on the child's safety and, if necessary, contact the police.

[0277] Example prompt for a generated AI model: "Design an anomaly detection model that uses a child's location data to detect deviations from their normal behavior patterns and notify parents. This model aims to monitor the route home from school and detect unnatural route changes in real time."

[0278] This system makes it easy for guardians to confirm the safety of children and take necessary measures.

[0279] The flow of the specific process in Example 1 will be described using FIG. 11.

[0280] Step 1:

[0281] The terminal functions as a wearable device worn by the child and uses a GPS sensor and an acceleration sensor to periodically (for example, every 5 seconds) obtain position information and movement data. The input is the latitude and longitude of the object and the speed and direction of movement, and the output is in the form of raw data. This data is temporarily stored in the terminal and is ready to be sent in batch form at regular intervals.

[0282] Step 2:

[0283] The terminal sends the collected batch-form position data to the server. The input is the position and movement data collected in Step 1, and the output is the batched data packets. The HTTPS protocol is used for data transmission to ensure secure and safe communication.

[0284] Step 3:

[0285] The server receives the position data sent from the terminal and stores it in the database. The input is the raw position data sent from the terminal, and the output is the stored data. Simultaneously with the storage, the server prepares to start data analysis based on the received data.

[0286] Step 4: <​The server runs machine learning algorithms to analyze the stored data. This analysis uses the scikit-learn library in Python to model typical behavioral patterns based on historical data. The input is historical location and movement data, and the output is a typical behavioral pattern model. This model serves as a benchmark for evaluating new data.

[0288] Step 5:

[0289] The server uses the model obtained from the analysis to evaluate how much the newly received data deviates from the normal range of movement. The input is the latest location data and behavior pattern model sent in step 2, and the output is a determination of whether or not an anomaly is present. If an anomaly is detected, that information is used in the next step.

[0290] Step 6:

[0291] The server immediately generates an alert upon detecting an anomaly. This alert includes information about the specific time and location where the anomaly was detected. The input is whether or not an anomaly was detected, as determined in step 5, and the output is the alert information. After the alert is generated, the notification system starts operating.

[0292] Step 7:

[0293] The server sends the generated alerts to the user's smart device via Firebase Cloud Messaging. The input is the generated alert information, and the output is a notification to the user. This notification allows the user to immediately understand their child's unusual behavior and consider necessary actions.

[0294] (Application Example 1)

[0295] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0296] Monitoring children's safety at home is a major challenge for parents. Especially in recent years, with the increase in dual-income households and heightened awareness of crime prevention, there is a growing need to understand any unusual situations within the home in real time and respond quickly. However, conventional safety monitoring systems lack sufficient coverage and functionality, making effective responses difficult.

[0297] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0298] In this invention, the server includes means for acquiring location information, means for controlling an image recording device when an anomaly is detected, and means for issuing a warning via voice output when an anomaly occurs. This enables monitoring of children's safety within the home in a variety of ways and allows for a rapid response.

[0299] "Means for acquiring location information" refers to technologies and devices for determining the current location of a moving object.

[0300] "Means for analyzing acquired location information" refers to functions and algorithms for evaluating behavioral patterns and anomalies based on the obtained location data.

[0301] "Means for modeling the normal range of movement" refers to a function that uses mathematical or statistical methods to represent the normal behavior and range of movement of a moving object obtained from repeated actions.

[0302] A "means for detecting anomalies" is a system that identifies discrepancies between a modeled normal range of movement and actual movements, and signals an abnormal situation.

[0303] "Means of generating alarms" refers to the process of generating signals or messages to quickly notify users when an anomaly is detected.

[0304] "Means of notifying users" refers to communication means or devices used to transmit alarms or information to users.

[0305] The "means for communicating with external institutions" refers to a system or technology for transmitting information to pre-determined external organizations or individuals.

[0306] The "means for monitoring a moving object in a home environment" refers to a mechanism for continuously observing and recording the movement of a moving object within a home.

[0307] The "means for controlling an image recording device" refers to an operation or method for managing the operation of a device that records images or videos in specific situations.

[0308] The "means for issuing a warning with a voice output means in case of an abnormality" refers to a method for transmitting warnings or instructions using voice when an abnormality occurs.

[0309] The system for implementing this invention operates using several main components to monitor the safety of children in a home environment. The server acquires location information and detects abnormalities by comparing it with the normally modeled behavior range. When an abnormality is detected, an alarm is generated and promptly notified to the user. Furthermore, it has functions for monitoring the situation within the home, controlling the monitoring of a moving object, and controlling an image recording device. In case of an abnormality, the user can be alerted using a voice output means.

[0310] As a specific system configuration, a wearable device measures the location information of a child in real time and transmits it to the server. The server analyzes the received location information and generates a normal behavior pattern using machine learning algorithms such as TensorFlow or PyTorch. When an abnormality is detected, the server controls an image recording device such as a camera or a recording device and simultaneously issues necessary warnings using a voice output device.

[0311] For example, if a child deviates from their usual range of movement, the server immediately detects the anomaly and begins recording images. It can also issue audio alerts and provide appropriate instructions to protect the child. This allows parents to immediately understand the situation via their smartphones and, if necessary, coordinate with external organizations.

[0312] A concrete example of using a generative AI model is the ability to generate a prompt message such as, "Develop a system that ensures children are playing safely at home and notifies the user if they go outside a defined area." In this way, the system can provide a safe home environment and offer overall support to ensure children's safety.

[0313] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0314] Step 1:

[0315] The device acquires the child's location information through a wearable device. It receives GPS information and acceleration data as input, formats them into a data format for transmission to the server, and sends the formatted data to the server as output.

[0316] Step 2:

[0317] The server analyzes location and acceleration data received from the terminal. It receives data from the terminal as input and analyzes normal behavior patterns using machine learning algorithms (e.g., TensorFlow). During the data processing process, it performs noise reduction and data smoothing, and generates normal behavior patterns as output.

[0318] Step 3:

[0319] The server analyzes newly received location information based on the generated normal behavior pattern and detects whether there is an anomaly. It takes new location information as input and compares it with the previous pattern. If an anomaly pattern is found, it sets an anomaly detection flag as output.

[0320] Step 4:

[0321] If an anomaly is detected, the server generates an alarm and notifies the user's smartphone of this information. It receives an anomaly detection flag as input and generates the notification content. The output is sent to the user in the form of a text message or push notification.

[0322] Step 5:

[0323] The server simultaneously controls the video recording device installed in the home and issues a command to record the situation when an anomaly is detected. It uses an anomaly detection flag as input to generate the recording command. The output is a control signal to the recording device.

[0324] Step 6:

[0325] The server operates an audio output device in the event of an anomaly, sending an audio message to alert the child. It uses alarm information as input to generate the audio message. The output is an audio signal sent to the audio device.

[0326] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0327] This invention is a system for protecting children's safety, and by taking user emotions into consideration, it provides more flexible and effective warnings and responses. This system operates primarily by utilizing wearable devices, servers, smartphone applications, and an emotion engine.

[0328] Operation in wearable devices

[0329] The device is designed as a wearable device for children and is equipped with a GPS sensor and an accelerometer. It collects the child's location information and movement patterns in real time and transmits them to a server. If the child deviates from their normal range of activity, this information is immediately recorded as abnormal data.

[0330] Server-based data analysis and the use of emotion engines

[0331] The server has the capability to analyze data received from the terminal and detect anomalies by comparing them to the user's normal range of activity. Furthermore, an emotion engine analyzes the user's current emotional state. Based on the results of this emotion analysis, the urgency and content of the alert are adjusted. For example, if the user is feeling excessively anxious, the notification may be changed to calmer language or support information may be added.

[0332] Generation of alarm notifications and coordination with external organizations

[0333] If abnormal behavior is detected, the server generates an alarm and sends it to the user. The way the alarm should be expressed is determined based on emotional information provided by the emotion engine. The user can also check the child's location and movement history through a smartphone app and decide whether to contact an external organization. If contact with an external organization is necessary, the communication will be made at an appropriate time and with appropriate content, based on the analysis results of the emotion engine.

[0334] Specific example

[0335] For example, if a child deviates from their usual route home and stops at a certain location for an extended period, the server detects this as an anomaly and notifies the parent. If the emotion engine analyzes the situation and determines that the parent is highly anxious, the notification will include a reassuring message. Furthermore, if the user chooses to contact the police, prompt and appropriate action will be required, taking into account the emotional information.

[0336] Thus, the present invention provides a system that can effectively protect children's safety by integrating data analysis, sentiment analysis, and rapid collaboration.

[0337] The following describes the processing flow.

[0338] Step 1:

[0339] The device uses a built-in GPS sensor to collect the child's current location at regular intervals. It also acquires movement data using an accelerometer and sends all of this data together to a server.

[0340] Step 2:

[0341] The server analyzes location and movement data received from the terminal and evaluates the current situation based on a normal behavior model. If abnormal movement is detected, a flag is set.

[0342] Step 3:

[0343] The server prompts the emotion engine to evaluate the user's current emotions, regardless of whether an anomaly flag is present. The emotion engine infers and analyzes emotions based on audio data, touch data, and other information obtained from the user's smartphone or wearable device.

[0344] Step 4:

[0345] The server adjusts the content of the alarms it generates when an anomaly is detected, based on the output of the emotion engine. If it determines that the user is experiencing anxiety, it makes adjustments such as softening the notification message.

[0346] Step 5:

[0347] Users check the alerts they receive on their smartphones, pinpoint their child's current location based on the information provided, and view their activity history within the app to understand the situation.

[0348] Step 6:

[0349] Users can contact external organizations such as the police through the app as needed. If the emotion engine determines that the user is in an anxious state, a guide will be added to the notification to help them contact external organizations more smoothly.

[0350] Step 7:

[0351] The device confirms that contact has been made with an external organization, continuously updates its location information in real time, and provides the latest information to the external organization via the server.

[0352] (Example 2)

[0353] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0354] Simply detecting anomalies and issuing alarms to ensure children's safety can cause excessive anxiety among parents. In such situations, cold, emotionless notifications can actually increase anxiety and hinder a swift and appropriate response. Therefore, alarm notifications that take the user's emotional state into consideration are necessary.

[0355] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0356] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, means for modeling the normal range of activity based on the analysis, means for detecting anomalies by comparing with the normal range of activity, means for generating an alarm when an anomaly is detected, means for notifying the user of the alarm, means for communicating with an external organization based on the alarm, means for analyzing the user's emotional state and adjusting the content of the alarm, and means for changing the urgency of the alarm based on emotional information when an anomaly is detected. This enables flexible and effective alarm notifications that respond to the user's emotional state, and is expected to allow for a quick and appropriate response to ensure the safety of children.

[0357] "Location information" refers to data that indicates the geographical location of a specific object or individual.

[0358] "Analysis" is the process of examining collected information and data in detail and extracting patterns and features.

[0359] "Normal range of movement" refers to the area and route that the subject is expected to normally travel, modeled based on past behavioral data.

[0360] "Anomaly detection" is the process of identifying movements that deviate from the normal, expected range of activity or behavioral patterns, and determining that these movements are abnormal.

[0361] An "alert" is a signal or message used to notify a user that attention or caution is necessary when specific conditions or situations occur.

[0362] An "external organization" refers to an organization or group that exists outside the system and with which communication and collaboration are possible.

[0363] "Emotional state" refers to the psychological state or emotional changes an individual is experiencing at a given time.

[0364] "Urgency" is a measure that indicates how quickly a response is required to a particular situation or event.

[0365] This invention is a system designed to ensure the safety of children, utilizing a wearable device, a server, a smartphone application, and an emotion engine. The invention proceeds roughly as follows:

[0366] Use of wearable devices

[0367] The device functions as a wearable device for children. It is equipped with a GPS sensor for acquiring location information and an accelerometer for detecting abnormal movement. The device collects location information and movement patterns in real time and transmits this data to a server. This data is then used to prepare for the detection of deviations from the child's normal range of activity.

[0368] Server Role

[0369] The server analyzes location information based on received data and models the user's normal range of movement. Furthermore, the server detects anomalies by comparing the current location to the normal range of movement. If an anomaly is detected, it generates an alarm and sends it to the user as a notification. In addition, the server is equipped with an emotion engine that analyzes the user's emotional state. Based on the results of this analysis, the urgency and content of the alarm message are adjusted.

[0370] User interaction

[0371] Users can receive alert notifications through a smartphone app and check their child's location and movement history in real time. If necessary, users can contact external organizations, and the system will assist in making such contacts at the optimal time and with the most appropriate content, based on emotional information.

[0372] As a concrete example, a user can input a prompt message to the AI ​​model saying, "When abnormal behavior in a child is detected, please generate a message that will reassure the parent," which will help generate emotionally sensitive messages.

[0373] This system ensures that each function works effectively together, enabling a quick and appropriate response to protect children's safety.

[0374] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0375] Step 1:

[0376] The device acquires location information and movement patterns via a wearable device worn by the child. Specifically, it measures location in real time using a GPS sensor and captures changes in movement using an accelerometer. This data is initially stored locally and then sent to a server. The input is sensor data from the wearable device, and the output is data transmission to the server.

[0377] Step 2:

[0378] The server receives data transmitted from the terminal. It analyzes location and acceleration data and compares it to past patterns to model the normal range of movement. This analysis prepares the server to detect deviations from normal behavior. The input is the data transmitted by the terminal, and the output is the result of the behavior pattern analysis. Based on these results, the server prepares the data for the next step.

[0379] Step 3:

[0380] The server detects anomalies by comparing the normal range of activity with the latest data. When an anomaly is detected, the server flags it as an anomaly and generates an alarm. Specifically, it determines the content and urgency of the alarm based on the type and severity of the abnormal behavior. The input is the result of the behavior pattern analysis, and the output is an instruction to generate an alarm.

[0381] Step 4:

[0382] The server analyzes the user's emotional state using an emotion engine. For example, it identifies anxiety and reassurance by considering alarms and conditioned responses the user has received in the past. This analysis result is used to determine the content and tone of the next alarm message to be generated. The input is interaction data with the user, and the output is the result of the emotional state analysis.

[0383] Step 5:

[0384] The server generates an appropriate alarm message based on the results of anomaly detection and sentiment analysis, and notifies the user. This message incorporates information from the sentiment engine. Upon receiving this notification, the user checks the details via a smartphone app and takes immediate action as needed. The input is the server's analysis result, and the output is the final alarm message sent to the user.

[0385] Step 6:

[0386] Users receive alert notifications through a smartphone app and can check their child's location and movement history. If a user determines that contacting an external agency is necessary, they can use the support prompts built into the app to make contact at the optimal time and with appropriate content, taking emotional information into consideration. The input is an alert message from the server, and the output is the user's action of contacting an external agency.

[0387] (Application Example 2)

[0388] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0389] Ensuring child safety is a critical issue in modern society. However, conventional location-based systems fail to consider the emotional state of parents when detecting anomalies and issuing alarms, potentially causing excessive anxiety. Therefore, there is a need for technology that enables flexible and effective responses that take parents' emotions into account when detecting anomalies and generating alarms based on location information.

[0390] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0391] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, and means for adjusting the content of the alarm based on the analyzed emotional state. This makes it possible to issue an appropriate alarm that provides reassurance, taking into account the emotional state of the guardian, when an abnormality in the child's safety is detected.

[0392] "Location information" refers to data that indicates the specific geographical location of an object.

[0393] "Means of analysis" refers to methods or devices for analyzing collected data and extracting useful information from that data.

[0394] "Normal range of activity" is a model of the geographical area that a subject typically travels within on a daily basis.

[0395] "Means for detecting anomalies" refers to methods or devices for identifying movements that deviate from normal behavioral patterns.

[0396] "Means for generating an alarm" refers to a method or device for creating an alert to draw attention when an anomaly is detected.

[0397] "User emotional state" refers to the psychological or emotional situation a user is currently experiencing.

[0398] "Means for adjusting the content of an alarm based on emotional state" refers to a method or apparatus for optimizing an alarm message in accordance with the user's emotional analysis.

[0399] "Means of notification" refers to a method or device for transmitting information or a message to a designated recipient.

[0400] "Means of communication with external organizations" refers to methods or devices that a system uses to exchange information with external organizations or services.

[0401] In the system implementing this invention, a wearable device first acquires the child's location information in real time and transmits it to a server. The wearable device is equipped with a GPS sensor and an accelerometer. The server analyzes the received location information and detects anomalies by comparing it with the child's normal range of activity. At this time, the server uses a machine learning algorithm to model the child's normal range of activity based on past behavioral patterns.

[0402] When an anomaly is detected, the server generates an alarm and notifies the user. Furthermore, it uses an emotion engine to analyze the user's emotional state. The emotion engine utilizes a generative AI model to analyze the user's emotions and provides a means to adjust the content of the alarm based on the results. For example, if the user is feeling anxious, the notification will include a reassuring message.

[0403] Users can check their child's current location and movement history through a smartphone application. Based on this information, they can determine whether it is necessary to contact an external agency. When communicating with an external agency, the server will send a notification based on emotional information at the appropriate time.

[0404] For example, if a child deviates from their usual route to school and is stopped for an extended period at an unexpected location, the server will consider this information suspicious and send a notification to the parent. The AI ​​model then adds considerate wording to the notification for anxious parents, such as, "Your child has been confirmed safe. Please remain calm and check the situation."

[0405] Examples of prompt messages include, "Generate a calm and reassuring notification message based on the user's emotions," and "Create a reassuring notification message for parents if abnormal behavior is detected."

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The device uses GPS and accelerometer sensors to acquire the child's location and movement data in real time. This data, including the child's current location and movement patterns, is transmitted from the device to the server. The input is location and velocity data from the sensors, and the output is raw data sent to the server.

[0409] Step 2:

[0410] The server analyzes the received location information and models the normal range of movement by comparing it with past location information. This process uses machine learning algorithms to generate normal behavior patterns and establish criteria for detecting anomalies. The input is location information from the terminal, and the output is a model of the normal range of movement.

[0411] Step 3:

[0412] The server detects an anomaly when location information deviates from the normal range of activity. Upon detection of an anomaly, it generates an alarm and proceeds to the next step. In this step, real-time location data is evaluated based on the generated normal range model to identify the anomaly. The input is real-time location information, and the output is an anomaly detection flag.

[0413] Step 4:

[0414] The server uses a generative AI model to analyze the user's emotions. In this analysis, it detects the user's current emotional state and adjusts the alarm content accordingly. The input is emotional data obtained from the user (e.g., questionnaires for emotion analysis or facial recognition data), and the output is the user's emotional state.

[0415] Step 5:

[0416] If an anomaly is detected, the server adjusts the alarm content based on the user's emotional state and sends a notification to the user's smartphone. This notification includes an emotionally sensitive message, aiming to provide a sense of reassurance. The input is the anomaly detection flag and the adjusted alarm content, and the output is the notification message to the user.

[0417] Step 6:

[0418] Users receive notifications and can use a smartphone application to check their child's location and movement history in real time. They can also consider contacting external organizations if necessary. Input is notification messages from the server, and output is a log of the user's decision-making.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] This invention is a system designed to ensure the safety of children, enabling rapid response by acquiring location information through a wearable device and detecting abnormal behavior. The device operates as follows:

[0436] Device-level operation

[0437] The device functions as a wearable device worn by children. It has a GPS sensor for individually acquiring location information and can also analyze detailed movement patterns using an accelerometer. The collected data is sent to a server in fixed batch sizes. The device has pre-set areas for the child's usual school route and activity range, and records and analyzes their movements within those areas.

[0438] Data analysis on the server

[0439] The server analyzes location and movement data received from the terminal. This allows it to model the normal behavioral patterns of each child, and an algorithm operates to detect abnormal movements in real time based on these criteria. Machine learning techniques are used in this analysis, and the model is continuously improved based on new data.

[0440] Anomaly detection and notification

[0441] When an anomaly is detected, the server immediately generates an alert and sends a notification to the user's (parent's) smartphone. This notification includes the location and time of the anomaly, as well as the current location information. Based on this, parents can take immediate action.

[0442] Collaboration with external organizations

[0443] If a parent or guardian notices an abnormal situation and determines that immediate action is necessary, the device has communication capabilities to connect with pre-configured external organizations, such as the police or local crime prevention groups. This allows for the transmission of the child's precise location and situation information to the relevant organizations, enabling prompt intervention.

[0444] For example, if a child deviates from their usual route home from school and stays in an unfamiliar location, the server detects the anomaly and immediately sends an alert to the parent. This alert allows the parent to call their child to check on their safety or, if necessary, notify the police. Through these processes, a comprehensive solution is provided to protect children's safety.

[0445] The following describes the processing flow.

[0446] Step 1:

[0447] The device uses its built-in GPS sensor to acquire the child's location information at regular intervals. This information is temporarily stored in the device's memory.

[0448] Step 2:

[0449] The device also collects data from its accelerometer and performs a basic check on the spot to analyze its movement pattern and detect any abnormalities. It then prepares to send the results, along with location information, to the server in batches.

[0450] Step 3:

[0451] After receiving location and movement data transmitted from the terminal, the server models the typical range of movement by comparing it with historical data.

[0452] Step 4:

[0453] The server compares the latest data with a normal behavior model and sets an anomaly detection flag if any behavior deviating from the standard is detected.

[0454] Step 5:

[0455] If an anomaly detection flag is set, the server immediately generates an alert and pushes the details to the user (parent / guardian)'s smartphone application.

[0456] Step 6:

[0457] Users can review received alerts and view their child's current location and movement history from the application interface.

[0458] Step 7:

[0459] If the user determines that the abnormality will persist, they can instruct the application to contact external organizations such as the police.

[0460] Step 8:

[0461] After being instructed to contact an external organization, the device continuously transmits location information to the server and performs the procedure of sending necessary information to the external organization while maintaining the latest information.

[0462] (Example 1)

[0463] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0464] In recent years, there has been a growing demand for ensuring the safety of children, and monitoring their location during school commutes and daily activities, as well as the early detection of abnormal behavior, have become crucial issues. However, conventional systems lacked sufficient real-time dynamic data analysis, sometimes resulting in delays in detecting anomalies. Furthermore, the rapid notification of detected anomalies and inadequate coordination with external organizations failed to alleviate the anxieties of parents and guardians.

[0465] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0466] In this invention, the server includes means for collecting location data, means for analyzing the collected location data, and means for generating a general range of movement based on the analysis. This makes it possible to monitor a child's location in real time, send notifications quickly when an anomaly is detected, and cooperate with external organizations as needed.

[0467] "Location data" refers to latitude and longitude information used to identify a physical location. It may also include information about time and speed.

[0468] "Analysis" refers to the process of analyzing collected data using statistical or algorithmic methods to identify meaningful patterns or anomalies.

[0469] "General range of activity" is a model that represents the normal range of activity for a particular individual or object, and is created based on past data.

[0470] "An anomaly" refers to a phenomenon that deviates from a predetermined normal range of behavior or behavioral patterns.

[0471] "Notification" refers to a system message that provides alerts or warnings to users or relevant organizations when an anomaly is detected.

[0472] "Information exchange with other organizations" refers to the procedure of coordinating with external organizations and groups as needed to share location information and detailed information about the situation when an anomaly is detected.

[0473] This invention is a location data monitoring system for ensuring the safety of children. The system consists of a wearable device, a server for data analysis, and a user's smart device for receiving alerts.

[0474] The device functions as a small wearable device worn by children, collecting location and movement data using a GPS sensor and an accelerometer. Specifically, a chip embedded in the device collects latitude and longitude information, as well as movement speed and acceleration, at regular intervals, and sends this data to a server in batch format. HTTPS is used as the communication protocol for data transmission to ensure data security.

[0475] The server analyzes the data received from the terminal using machine learning algorithms. Preferably, the analysis is performed using the Python library scikit-learn to model normal behavioral patterns. Based on the model, the server evaluates the current data and immediately generates an alert if an anomaly is detected. This alert is sent to the user's smart device via the Firebase Cloud Messaging service.

[0476] Users receive alerts via their smartphones. These alerts include the specific time and location where an anomaly was detected, enabling a rapid response. Through the application's features, users can, if necessary, send information to external organizations such as the police or local crime prevention groups to request a swift response.

[0477] For example, if a child takes an unusual route home from school, the server will detect the anomaly and immediately notify the parent. The parent can then check on the child's safety and, if necessary, contact the police.

[0478] Example prompt for a generated AI model: "Design an anomaly detection model that uses a child's location data to detect deviations from their normal behavior patterns and notify parents. This model aims to monitor the route home from school and detect unnatural route changes in real time."

[0479] This system makes it easier for parents to ensure their children's safety and take necessary measures.

[0480] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0481] Step 1:

[0482] The device functions as a wearable device worn by children and periodically acquires location and movement data (e.g., every 5 seconds) using GPS and accelerometer sensors. Inputs include the object's latitude and longitude, as well as its speed and direction of movement, while output is in raw data format. This data is temporarily stored within the device and prepared for transmission in batches at regular intervals.

[0483] Step 2:

[0484] The terminal sends the collected location data in batch format to the server. The input is the location and movement data collected in step 1, and the output is batched data packets. Secure and safe communication is ensured by using the HTTPS protocol for data transmission.

[0485] Step 3:

[0486] The server receives location data transmitted from the terminal and stores it in a database. The input is the raw location data transmitted from the terminal, and the output is the stored data. Simultaneously with saving, the server prepares to begin data analysis based on the received data.

[0487] Step 4:

[0488] The server runs machine learning algorithms to analyze the stored data. This analysis uses the scikit-learn library in Python to model typical behavioral patterns based on historical data. The input is historical location and movement data, and the output is a typical behavioral pattern model. This model serves as a benchmark for evaluating new data.

[0489] Step 5:

[0490] The server uses the model obtained from the analysis to evaluate how much the newly received data deviates from the normal range of movement. The input is the latest location data and behavior pattern model sent in step 2, and the output is a determination of whether or not an anomaly is present. If an anomaly is detected, that information is used in the next step.

[0491] Step 6:

[0492] The server immediately generates an alert upon detecting an anomaly. This alert includes information about the specific time and location where the anomaly was detected. The input is whether or not an anomaly was detected, as determined in step 5, and the output is the alert information. After the alert is generated, the notification system starts operating.

[0493] Step 7:

[0494] The server sends the generated alerts to the user's smart device via Firebase Cloud Messaging. The input is the generated alert information, and the output is a notification to the user. This notification allows the user to immediately understand their child's unusual behavior and consider necessary actions.

[0495] (Application Example 1)

[0496] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0497] Monitoring children's safety at home is a major challenge for parents. Especially in recent years, with the increase in dual-income households and heightened awareness of crime prevention, there is a growing need to understand any unusual situations within the home in real time and respond quickly. However, conventional safety monitoring systems lack sufficient coverage and functionality, making effective responses difficult.

[0498] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0499] In this invention, the server includes means for acquiring location information, means for controlling an image recording device when an anomaly is detected, and means for issuing a warning via voice output when an anomaly occurs. This enables monitoring of children's safety within the home in a variety of ways and allows for a rapid response.

[0500] "Means for acquiring location information" refers to technologies and devices for determining the current location of a moving object.

[0501] "Means for analyzing acquired location information" refers to functions and algorithms for evaluating behavioral patterns and anomalies based on the obtained location data.

[0502] "Means for modeling the normal range of movement" refers to a function that uses mathematical or statistical methods to represent the normal behavior and range of movement of a moving object obtained from repeated actions.

[0503] A "means for detecting anomalies" is a system that identifies discrepancies between a modeled normal range of movement and actual movements, and signals an abnormal situation.

[0504] "Means of generating alarms" refers to the process of generating signals or messages to quickly notify users when an anomaly is detected.

[0505] "Means of notifying users" refers to communication means or devices used to transmit alarms or information to users.

[0506] "Means of communication with external organizations" refers to systems and technologies for transmitting information to predetermined external organizations or individuals.

[0507] "Means for monitoring moving objects in a home environment" refers to a system for continuously observing and recording the movements of moving objects within a home.

[0508] "Means for controlling an image recording device" refers to operations or methods for managing the operation of a device that records images or videos under specific circumstances.

[0509] "Means for issuing warnings via voice output in the event of an abnormality" refers to methods for conveying warnings or instructions using voice when an abnormality occurs.

[0510] The system for implementing this invention operates using several key components to monitor the safety of children in a home environment. The server acquires location information and detects anomalies by comparing it with a modeled normal range of activity. If an anomaly is detected, it generates an alarm and promptly notifies the user. Furthermore, it has the function of controlling mobile monitoring and image recording devices to monitor the situation within the home. In the event of an anomaly, the user can be alerted using an audio output means.

[0511] The specific system configuration involves a wearable device measuring the child's location in real time and transmitting it to a server. The server analyzes the received location information and generates normal behavior patterns using machine learning algorithms such as TensorFlow and PyTorch. When an anomaly is detected, the server controls image recording devices such as cameras and video recorders, and simultaneously issues necessary warnings using an audio output device.

[0512] For example, if a child deviates from their usual range of movement, the server immediately detects the anomaly and begins recording images. It can also issue audio alerts and provide appropriate instructions to protect the child. This allows parents to immediately understand the situation via their smartphones and, if necessary, coordinate with external organizations.

[0513] A concrete example of using a generative AI model is the ability to generate a prompt message such as, "Develop a system that ensures children are playing safely at home and notifies the user if they go outside a defined area." In this way, the system can provide a safe home environment and offer overall support to ensure children's safety.

[0514] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0515] Step 1:

[0516] The device acquires the child's location information through a wearable device. It receives GPS information and acceleration data as input, formats them into a data format for transmission to the server, and sends the formatted data to the server as output.

[0517] Step 2:

[0518] The server analyzes location and acceleration data received from the terminal. It receives data from the terminal as input and analyzes normal behavior patterns using machine learning algorithms (e.g., TensorFlow). During the data processing process, it performs noise reduction and data smoothing, and generates normal behavior patterns as output.

[0519] Step 3:

[0520] The server analyzes newly received location information based on the generated normal behavior pattern and detects whether there is an anomaly. It takes new location information as input and compares it with the previous pattern. If an anomaly pattern is found, it sets an anomaly detection flag as output.

[0521] Step 4:

[0522] If an anomaly is detected, the server generates an alarm and notifies the user's smartphone of this information. It receives an anomaly detection flag as input and generates the notification content. The output is sent to the user in the form of a text message or push notification.

[0523] Step 5:

[0524] The server simultaneously controls the video recording device installed in the home and issues a command to record the situation when an anomaly is detected. It uses an anomaly detection flag as input to generate the recording command. The output is a control signal to the recording device.

[0525] Step 6:

[0526] The server operates an audio output device in the event of an anomaly, sending an audio message to alert the child. It uses alarm information as input to generate the audio message. The output is an audio signal sent to the audio device.

[0527] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0528] This invention is a system for protecting children's safety, and by taking user emotions into consideration, it provides more flexible and effective warnings and responses. This system operates primarily by utilizing wearable devices, servers, smartphone applications, and an emotion engine.

[0529] Operation in wearable devices

[0530] The device is designed as a wearable device for children and is equipped with a GPS sensor and an accelerometer. It collects the child's location information and movement patterns in real time and transmits them to a server. If the child deviates from their normal range of activity, this information is immediately recorded as abnormal data.

[0531] Server-based data analysis and the use of emotion engines

[0532] The server has the capability to analyze data received from the terminal and detect anomalies by comparing them to the user's normal range of activity. Furthermore, an emotion engine analyzes the user's current emotional state. Based on the results of this emotion analysis, the urgency and content of the alert are adjusted. For example, if the user is feeling excessively anxious, the notification may be changed to calmer language or support information may be added.

[0533] Generation of alarm notifications and coordination with external organizations

[0534] If abnormal behavior is detected, the server generates an alarm and sends it to the user. The way the alarm should be expressed is determined based on emotional information provided by the emotion engine. The user can also check the child's location and movement history through a smartphone app and decide whether to contact an external organization. If contact with an external organization is necessary, the communication will be made at an appropriate time and with appropriate content, based on the analysis results of the emotion engine.

[0535] Specific example

[0536] For example, if a child deviates from their usual route home and stops at a certain location for an extended period, the server detects this as an anomaly and notifies the parent. If the emotion engine analyzes the situation and determines that the parent is highly anxious, the notification will include a reassuring message. Furthermore, if the user chooses to contact the police, prompt and appropriate action will be required, taking into account the emotional information.

[0537] Thus, the present invention provides a system that can effectively protect children's safety by integrating data analysis, sentiment analysis, and rapid collaboration.

[0538] The following describes the processing flow.

[0539] Step 1:

[0540] The device uses a built-in GPS sensor to collect the child's current location at regular intervals. It also acquires movement data using an accelerometer and sends all of this data together to a server.

[0541] Step 2:

[0542] The server analyzes location and movement data received from the terminal and evaluates the current situation based on a normal behavior model. If abnormal movement is detected, a flag is set.

[0543] Step 3:

[0544] The server prompts the emotion engine to evaluate the user's current emotions, regardless of whether an anomaly flag is present. The emotion engine infers and analyzes emotions based on audio data, touch data, and other information obtained from the user's smartphone or wearable device.

[0545] Step 4:

[0546] The server adjusts the content of the alarms it generates when an anomaly is detected, based on the output of the emotion engine. If it determines that the user is experiencing anxiety, it makes adjustments such as softening the notification message.

[0547] Step 5:

[0548] Users check the alerts they receive on their smartphones, pinpoint their child's current location based on the information provided, and view their activity history within the app to understand the situation.

[0549] Step 6:

[0550] Users can contact external organizations such as the police through the app as needed. If the emotion engine determines that the user is in an anxious state, a guide will be added to the notification to help them contact external organizations more smoothly.

[0551] Step 7:

[0552] The device confirms that contact has been made with an external organization, continuously updates its location information in real time, and provides the latest information to the external organization via the server.

[0553] (Example 2)

[0554] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0555] Simply detecting anomalies and issuing alarms to ensure children's safety can cause excessive anxiety among parents. In such situations, cold, emotionless notifications can actually increase anxiety and hinder a swift and appropriate response. Therefore, alarm notifications that take the user's emotional state into consideration are necessary.

[0556] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0557] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, means for modeling the normal range of activity based on the analysis, means for detecting anomalies by comparing with the normal range of activity, means for generating an alarm when an anomaly is detected, means for notifying the user of the alarm, means for communicating with an external organization based on the alarm, means for analyzing the user's emotional state and adjusting the content of the alarm, and means for changing the urgency of the alarm based on emotional information when an anomaly is detected. This enables flexible and effective alarm notifications that respond to the user's emotional state, and is expected to allow for a quick and appropriate response to ensure the safety of children.

[0558] "Location information" refers to data that indicates the geographical location of a specific object or individual.

[0559] "Analysis" is the process of examining collected information and data in detail and extracting patterns and features.

[0560] "Normal range of movement" refers to the area and route that the subject is expected to normally travel, modeled based on past behavioral data.

[0561] "Anomaly detection" is the process of identifying movements that deviate from the normal, expected range of activity or behavioral patterns, and determining that these movements are abnormal.

[0562] An "alert" is a signal or message used to notify a user that attention or caution is necessary when specific conditions or situations occur.

[0563] An "external organization" refers to an organization or group that exists outside the system and with which communication and collaboration are possible.

[0564] "Emotional state" refers to the psychological state or emotional changes an individual is experiencing at a given time.

[0565] "Urgency" is a measure that indicates how quickly a response is required to a particular situation or event.

[0566] This invention is a system designed to ensure the safety of children, utilizing a wearable device, a server, a smartphone application, and an emotion engine. The invention proceeds roughly as follows:

[0567] Use of wearable devices

[0568] The device functions as a wearable device for children. It is equipped with a GPS sensor for acquiring location information and an accelerometer for detecting abnormal movement. The device collects location information and movement patterns in real time and transmits this data to a server. This data is then used to prepare for the detection of deviations from the child's normal range of activity.

[0569] Server Role

[0570] The server analyzes location information based on received data and models the user's normal range of movement. Furthermore, the server detects anomalies by comparing the current location to the normal range of movement. If an anomaly is detected, it generates an alarm and sends it to the user as a notification. In addition, the server is equipped with an emotion engine that analyzes the user's emotional state. Based on the results of this analysis, the urgency and content of the alarm message are adjusted.

[0571] User interaction

[0572] Users can receive alert notifications through a smartphone app and check their child's location and movement history in real time. If necessary, users can contact external organizations, and the system will assist in making such contacts at the optimal time and with the most appropriate content, based on emotional information.

[0573] As a concrete example, a user can input a prompt message to the AI ​​model saying, "When abnormal behavior in a child is detected, please generate a message that will reassure the parent," which will help generate emotionally sensitive messages.

[0574] This system ensures that each function works effectively together, enabling a quick and appropriate response to protect children's safety.

[0575] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0576] Step 1:

[0577] The device acquires location information and movement patterns via a wearable device worn by the child. Specifically, it measures location in real time using a GPS sensor and captures changes in movement using an accelerometer. This data is initially stored locally and then sent to a server. The input is sensor data from the wearable device, and the output is data transmission to the server.

[0578] Step 2:

[0579] The server receives data transmitted from the terminal. It analyzes location and acceleration data and compares it to past patterns to model the normal range of movement. This analysis prepares the server to detect deviations from normal behavior. The input is the data transmitted by the terminal, and the output is the result of the behavior pattern analysis. Based on these results, the server prepares the data for the next step.

[0580] Step 3:

[0581] The server detects anomalies by comparing the normal range of activity with the latest data. When an anomaly is detected, the server flags it as an anomaly and generates an alarm. Specifically, it determines the content and urgency of the alarm based on the type and severity of the abnormal behavior. The input is the result of the behavior pattern analysis, and the output is an instruction to generate an alarm.

[0582] Step 4:

[0583] The server analyzes the user's emotional state using an emotion engine. For example, it identifies anxiety and reassurance by considering alarms and conditioned responses the user has received in the past. This analysis result is used to determine the content and tone of the next alarm message to be generated. The input is interaction data with the user, and the output is the result of the emotional state analysis.

[0584] Step 5:

[0585] The server generates an appropriate alarm message based on the results of anomaly detection and sentiment analysis, and notifies the user. This message incorporates information from the sentiment engine. Upon receiving this notification, the user checks the details via a smartphone app and takes immediate action as needed. The input is the server's analysis result, and the output is the final alarm message sent to the user.

[0586] Step 6:

[0587] Users receive alert notifications through a smartphone app and can check their child's location and movement history. If a user determines that contacting an external agency is necessary, they can use the support prompts built into the app to make contact at the optimal time and with appropriate content, taking emotional information into consideration. The input is an alert message from the server, and the output is the user's action of contacting an external agency.

[0588] (Application Example 2)

[0589] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0590] Ensuring child safety is a critical issue in modern society. However, conventional location-based systems fail to consider the emotional state of parents when detecting anomalies and issuing alarms, potentially causing excessive anxiety. Therefore, there is a need for technology that enables flexible and effective responses that take parents' emotions into account when detecting anomalies and generating alarms based on location information.

[0591] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0592] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, and means for adjusting the content of the alarm based on the analyzed emotional state. This makes it possible to issue an appropriate alarm that provides reassurance, taking into account the emotional state of the guardian, when an abnormality in the child's safety is detected.

[0593] "Location information" refers to data that indicates the specific geographical location of an object.

[0594] "Means of analysis" refers to methods or devices for analyzing collected data and extracting useful information from that data.

[0595] "Normal range of activity" is a model of the geographical area that a subject typically travels within on a daily basis.

[0596] "Means for detecting anomalies" refers to methods or devices for identifying movements that deviate from normal behavioral patterns.

[0597] "Means for generating an alarm" refers to a method or device for creating an alert to draw attention when an anomaly is detected.

[0598] "User emotional state" refers to the psychological or emotional situation a user is currently experiencing.

[0599] "Means for adjusting the content of an alarm based on emotional state" refers to a method or apparatus for optimizing an alarm message in accordance with the user's emotional analysis.

[0600] "Means of notification" refers to a method or device for transmitting information or a message to a designated recipient.

[0601] "Means of communication with external organizations" refers to methods or devices that a system uses to exchange information with external organizations or services.

[0602] In the system implementing this invention, a wearable device first acquires the child's location information in real time and transmits it to a server. The wearable device is equipped with a GPS sensor and an accelerometer. The server analyzes the received location information and detects anomalies by comparing it with the child's normal range of activity. At this time, the server uses a machine learning algorithm to model the child's normal range of activity based on past behavioral patterns.

[0603] When an anomaly is detected, the server generates an alarm and notifies the user. Furthermore, it uses an emotion engine to analyze the user's emotional state. The emotion engine utilizes a generative AI model to analyze the user's emotions and provides a means to adjust the content of the alarm based on the results. For example, if the user is feeling anxious, the notification will include a reassuring message.

[0604] Users can check their child's current location and movement history through a smartphone application. Based on this information, they can determine whether it is necessary to contact an external agency. When communicating with an external agency, the server will send a notification based on emotional information at the appropriate time.

[0605] For example, if a child deviates from their usual route to school and is stopped for an extended period at an unexpected location, the server will consider this information suspicious and send a notification to the parent. The AI ​​model then adds considerate wording to the notification for anxious parents, such as, "Your child has been confirmed safe. Please remain calm and check the situation."

[0606] Examples of prompt messages include, "Generate a calm and reassuring notification message based on the user's emotions," and "Create a reassuring notification message for parents if abnormal behavior is detected."

[0607] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0608] Step 1:

[0609] The device uses GPS and accelerometer sensors to acquire the child's location and movement data in real time. This data, including the child's current location and movement patterns, is transmitted from the device to the server. The input is location and velocity data from the sensors, and the output is raw data sent to the server.

[0610] Step 2:

[0611] The server analyzes the received location information and models the normal range of movement by comparing it with past location information. This process uses machine learning algorithms to generate normal behavior patterns and establish criteria for detecting anomalies. The input is location information from the terminal, and the output is a model of the normal range of movement.

[0612] Step 3:

[0613] The server detects an anomaly when location information deviates from the normal range of activity. Upon detection of an anomaly, it generates an alarm and proceeds to the next step. In this step, real-time location data is evaluated based on the generated normal range model to identify the anomaly. The input is real-time location information, and the output is an anomaly detection flag.

[0614] Step 4:

[0615] The server uses a generative AI model to analyze the user's emotions. In this analysis, it detects the user's current emotional state and adjusts the alarm content accordingly. The input is emotional data obtained from the user (e.g., questionnaires for emotion analysis or facial recognition data), and the output is the user's emotional state.

[0616] Step 5:

[0617] If an anomaly is detected, the server adjusts the alarm content based on the user's emotional state and sends a notification to the user's smartphone. This notification includes an emotionally sensitive message, aiming to provide a sense of reassurance. The input is the anomaly detection flag and the adjusted alarm content, and the output is the notification message to the user.

[0618] Step 6:

[0619] Users receive notifications and can use a smartphone application to check their child's location and movement history in real time. They can also consider contacting external organizations if necessary. Input is notification messages from the server, and output is a log of the user's decision-making.

[0620] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0621] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0622] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0623] [Fourth Embodiment]

[0624] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0625] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0626] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0627] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0628] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0629] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0630] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0631] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0632] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0633] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0634] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0635] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0636] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0637] This invention is a system designed to ensure the safety of children, enabling rapid response by acquiring location information through a wearable device and detecting abnormal behavior. The device operates as follows:

[0638] Device-level operation

[0639] The device functions as a wearable device worn by children. It has a GPS sensor for individually acquiring location information and can also analyze detailed movement patterns using an accelerometer. The collected data is sent to a server in fixed batch sizes. The device has pre-set areas for the child's usual school route and activity range, and records and analyzes their movements within those areas.

[0640] Data analysis on the server

[0641] The server analyzes location and movement data received from the terminal. This allows it to model the normal behavioral patterns of each child, and an algorithm operates to detect abnormal movements in real time based on these criteria. Machine learning techniques are used in this analysis, and the model is continuously improved based on new data.

[0642] Anomaly detection and notification

[0643] When an anomaly is detected, the server immediately generates an alert and sends a notification to the user's (parent's) smartphone. This notification includes the location and time of the anomaly, as well as the current location information. Based on this, parents can take immediate action.

[0644] Collaboration with external organizations

[0645] If a parent or guardian notices an abnormal situation and determines that immediate action is necessary, the device has communication capabilities to connect with pre-configured external organizations, such as the police or local crime prevention groups. This allows for the transmission of the child's precise location and situation information to the relevant organizations, enabling prompt intervention.

[0646] For example, if a child deviates from their usual route home from school and stays in an unfamiliar location, the server detects the anomaly and immediately sends an alert to the parent. This alert allows the parent to call their child to check on their safety or, if necessary, notify the police. Through these processes, a comprehensive solution is provided to protect children's safety.

[0647] The following describes the processing flow.

[0648] Step 1:

[0649] The device uses its built-in GPS sensor to acquire the child's location information at regular intervals. This information is temporarily stored in the device's memory.

[0650] Step 2:

[0651] The device also collects data from its accelerometer and performs a basic check on the spot to analyze its movement pattern and detect any abnormalities. It then prepares to send the results, along with location information, to the server in batches.

[0652] Step 3:

[0653] After receiving location and movement data transmitted from the terminal, the server models the typical range of movement by comparing it with historical data.

[0654] Step 4:

[0655] The server compares the latest data with a normal behavior model and sets an anomaly detection flag if any behavior deviating from the standard is detected.

[0656] Step 5:

[0657] If an anomaly detection flag is set, the server immediately generates an alert and pushes the details to the user (parent / guardian)'s smartphone application.

[0658] Step 6:

[0659] Users can review received alerts and view their child's current location and movement history from the application interface.

[0660] Step 7:

[0661] If the user determines that the abnormality will persist, they can instruct the application to contact external organizations such as the police.

[0662] Step 8:

[0663] After being instructed to contact an external organization, the device continuously transmits location information to the server and performs the procedure of sending necessary information to the external organization while maintaining the latest information.

[0664] (Example 1)

[0665] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0666] In recent years, there has been a growing demand for ensuring the safety of children, and monitoring their location during school commutes and daily activities, as well as the early detection of abnormal behavior, have become crucial issues. However, conventional systems lacked sufficient real-time dynamic data analysis, sometimes resulting in delays in detecting anomalies. Furthermore, the rapid notification of detected anomalies and inadequate coordination with external organizations failed to alleviate the anxieties of parents and guardians.

[0667] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0668] In this invention, the server includes means for collecting location data, means for analyzing the collected location data, and means for generating a general range of movement based on the analysis. This makes it possible to monitor a child's location in real time, send notifications quickly when an anomaly is detected, and cooperate with external organizations as needed.

[0669] "Location data" refers to latitude and longitude information used to identify a physical location. It may also include information about time and speed.

[0670] "Analysis" refers to the process of analyzing collected data using statistical or algorithmic methods to identify meaningful patterns or anomalies.

[0671] "General range of activity" is a model that represents the normal range of activity for a particular individual or object, and is created based on past data.

[0672] "An anomaly" refers to a phenomenon that deviates from a predetermined normal range of behavior or behavioral patterns.

[0673] "Notification" refers to a system message that provides alerts or warnings to users or relevant organizations when an anomaly is detected.

[0674] "Information exchange with other organizations" refers to the procedure of coordinating with external organizations and groups as needed to share location information and detailed information about the situation when an anomaly is detected.

[0675] This invention is a location data monitoring system for ensuring the safety of children. The system consists of a wearable device, a server for data analysis, and a user's smart device for receiving alerts.

[0676] The device functions as a small wearable device worn by children, collecting location and movement data using a GPS sensor and an accelerometer. Specifically, a chip embedded in the device collects latitude and longitude information, as well as movement speed and acceleration, at regular intervals, and sends this data to a server in batch format. HTTPS is used as the communication protocol for data transmission to ensure data security.

[0677] The server analyzes the data received from the terminal using machine learning algorithms. Preferably, the analysis is performed using the Python library scikit-learn to model normal behavioral patterns. Based on the model, the server evaluates the current data and immediately generates an alert if an anomaly is detected. This alert is sent to the user's smart device via the Firebase Cloud Messaging service.

[0678] Users receive alerts via their smartphones. These alerts include the specific time and location where an anomaly was detected, enabling a rapid response. Through the application's features, users can, if necessary, send information to external organizations such as the police or local crime prevention groups to request a swift response.

[0679] For example, if a child takes an unusual route home from school, the server will detect the anomaly and immediately notify the parent. The parent can then check on the child's safety and, if necessary, contact the police.

[0680] Example prompt for a generated AI model: "Design an anomaly detection model that uses a child's location data to detect deviations from their normal behavior patterns and notify parents. This model aims to monitor the route home from school and detect unnatural route changes in real time."

[0681] This system makes it easier for parents to ensure their children's safety and take necessary measures.

[0682] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0683] Step 1:

[0684] The device functions as a wearable device worn by children and periodically acquires location and movement data (e.g., every 5 seconds) using GPS and accelerometer sensors. Inputs include the object's latitude and longitude, as well as its speed and direction of movement, while output is in raw data format. This data is temporarily stored within the device and prepared for transmission in batches at regular intervals.

[0685] Step 2:

[0686] The terminal sends the collected location data in batch format to the server. The input is the location and movement data collected in step 1, and the output is batched data packets. Secure and safe communication is ensured by using the HTTPS protocol for data transmission.

[0687] Step 3:

[0688] The server receives location data transmitted from the terminal and stores it in a database. The input is the raw location data transmitted from the terminal, and the output is the stored data. Simultaneously with saving, the server prepares to begin data analysis based on the received data.

[0689] Step 4:

[0690] The server runs machine learning algorithms to analyze the stored data. This analysis uses the scikit-learn library in Python to model typical behavioral patterns based on historical data. The input is historical location and movement data, and the output is a typical behavioral pattern model. This model serves as a benchmark for evaluating new data.

[0691] Step 5:

[0692] The server uses the model obtained from the analysis to evaluate how much the newly received data deviates from the normal range of movement. The input is the latest location data and behavior pattern model sent in step 2, and the output is a determination of whether or not an anomaly is present. If an anomaly is detected, that information is used in the next step.

[0693] Step 6:

[0694] The server immediately generates an alert upon detecting an anomaly. This alert includes information about the specific time and location where the anomaly was detected. The input is whether or not an anomaly was detected, as determined in step 5, and the output is the alert information. After the alert is generated, the notification system starts operating.

[0695] Step 7:

[0696] The server sends the generated alerts to the user's smart device via Firebase Cloud Messaging. The input is the generated alert information, and the output is a notification to the user. This notification allows the user to immediately understand their child's unusual behavior and consider necessary actions.

[0697] (Application Example 1)

[0698] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0699] Monitoring children's safety at home is a major challenge for parents. Especially in recent years, with the increase in dual-income households and heightened awareness of crime prevention, there is a growing need to understand any unusual situations within the home in real time and respond quickly. However, conventional safety monitoring systems lack sufficient coverage and functionality, making effective responses difficult.

[0700] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0701] In this invention, the server includes means for acquiring location information, means for controlling an image recording device when an anomaly is detected, and means for issuing a warning via voice output when an anomaly occurs. This enables monitoring of children's safety within the home in a variety of ways and allows for a rapid response.

[0702] "Means for acquiring location information" refers to technologies and devices for determining the current location of a moving object.

[0703] "Means for analyzing acquired location information" refers to functions and algorithms for evaluating behavioral patterns and anomalies based on the obtained location data.

[0704] "Means for modeling the normal range of movement" refers to a function that uses mathematical or statistical methods to represent the normal behavior and range of movement of a moving object obtained from repeated actions.

[0705] A "means for detecting anomalies" is a system that identifies discrepancies between a modeled normal range of movement and actual movements, and signals an abnormal situation.

[0706] "Means of generating alarms" refers to the process of generating signals or messages to quickly notify users when an anomaly is detected.

[0707] "Means of notifying users" refers to communication means or devices used to transmit alarms or information to users.

[0708] "Means of communication with external organizations" refers to systems and technologies for transmitting information to predetermined external organizations or individuals.

[0709] "Means for monitoring moving objects in a home environment" refers to a system for continuously observing and recording the movements of moving objects within a home.

[0710] "Means for controlling an image recording device" refers to operations or methods for managing the operation of a device that records images or videos under specific circumstances.

[0711] "Means for issuing warnings via voice output in the event of an abnormality" refers to methods for conveying warnings or instructions using voice when an abnormality occurs.

[0712] The system for implementing this invention operates using several key components to monitor the safety of children in a home environment. The server acquires location information and detects anomalies by comparing it with a modeled normal range of activity. If an anomaly is detected, it generates an alarm and promptly notifies the user. Furthermore, it has the function of controlling mobile monitoring and image recording devices to monitor the situation within the home. In the event of an anomaly, the user can be alerted using an audio output means.

[0713] The specific system configuration involves a wearable device measuring the child's location in real time and transmitting it to a server. The server analyzes the received location information and generates normal behavior patterns using machine learning algorithms such as TensorFlow and PyTorch. When an anomaly is detected, the server controls image recording devices such as cameras and video recorders, and simultaneously issues necessary warnings using an audio output device.

[0714] For example, if a child deviates from their usual range of movement, the server immediately detects the anomaly and begins recording images. It can also issue audio alerts and provide appropriate instructions to protect the child. This allows parents to immediately understand the situation via their smartphones and, if necessary, coordinate with external organizations.

[0715] A concrete example of using a generative AI model is the ability to generate a prompt message such as, "Develop a system that ensures children are playing safely at home and notifies the user if they go outside a defined area." In this way, the system can provide a safe home environment and offer overall support to ensure children's safety.

[0716] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0717] Step 1:

[0718] The device acquires the child's location information through a wearable device. It receives GPS information and acceleration data as input, formats them into a data format for transmission to the server, and sends the formatted data to the server as output.

[0719] Step 2:

[0720] The server analyzes location and acceleration data received from the terminal. It receives data from the terminal as input and analyzes normal behavior patterns using machine learning algorithms (e.g., TensorFlow). During the data processing process, it performs noise reduction and data smoothing, and generates normal behavior patterns as output.

[0721] Step 3:

[0722] The server analyzes newly received location information based on the generated normal behavior pattern and detects whether there is an anomaly. It takes new location information as input and compares it with the previous pattern. If an anomaly pattern is found, it sets an anomaly detection flag as output.

[0723] Step 4:

[0724] If an anomaly is detected, the server generates an alarm and notifies the user's smartphone of this information. It receives an anomaly detection flag as input and generates the notification content. The output is sent to the user in the form of a text message or push notification.

[0725] Step 5:

[0726] The server simultaneously controls the video recording device installed in the home and issues a command to record the situation when an anomaly is detected. It uses an anomaly detection flag as input to generate the recording command. The output is a control signal to the recording device.

[0727] Step 6:

[0728] The server operates an audio output device in the event of an anomaly, sending an audio message to alert the child. It uses alarm information as input to generate the audio message. The output is an audio signal sent to the audio device.

[0729] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0730] This invention is a system for protecting children's safety, and by taking user emotions into consideration, it provides more flexible and effective warnings and responses. This system operates primarily by utilizing wearable devices, servers, smartphone applications, and an emotion engine.

[0731] Operation in wearable devices

[0732] The device is designed as a wearable device for children and is equipped with a GPS sensor and an accelerometer. It collects the child's location information and movement patterns in real time and transmits them to a server. If the child deviates from their normal range of activity, this information is immediately recorded as abnormal data.

[0733] Server-based data analysis and the use of emotion engines

[0734] The server has the capability to analyze data received from the terminal and detect anomalies by comparing them to the user's normal range of activity. Furthermore, an emotion engine analyzes the user's current emotional state. Based on the results of this emotion analysis, the urgency and content of the alert are adjusted. For example, if the user is feeling excessively anxious, the notification may be changed to calmer language or support information may be added.

[0735] Generation of alarm notifications and coordination with external organizations

[0736] If abnormal behavior is detected, the server generates an alarm and sends it to the user. The way the alarm should be expressed is determined based on emotional information provided by the emotion engine. The user can also check the child's location and movement history through a smartphone app and decide whether to contact an external organization. If contact with an external organization is necessary, the communication will be made at an appropriate time and with appropriate content, based on the analysis results of the emotion engine.

[0737] Specific example

[0738] For example, if a child deviates from their usual route home and stops at a certain location for an extended period, the server detects this as an anomaly and notifies the parent. If the emotion engine analyzes the situation and determines that the parent is highly anxious, the notification will include a reassuring message. Furthermore, if the user chooses to contact the police, prompt and appropriate action will be required, taking into account the emotional information.

[0739] Thus, the present invention provides a system that can effectively protect children's safety by integrating data analysis, sentiment analysis, and rapid collaboration.

[0740] The following describes the processing flow.

[0741] Step 1:

[0742] The device uses a built-in GPS sensor to collect the child's current location at regular intervals. It also acquires movement data using an accelerometer and sends all of this data together to a server.

[0743] Step 2:

[0744] The server analyzes location and movement data received from the terminal and evaluates the current situation based on a normal behavior model. If abnormal movement is detected, a flag is set.

[0745] Step 3:

[0746] The server prompts the emotion engine to evaluate the user's current emotions, regardless of whether an anomaly flag is present. The emotion engine infers and analyzes emotions based on audio data, touch data, and other information obtained from the user's smartphone or wearable device.

[0747] Step 4:

[0748] The server adjusts the content of the alarms it generates when an anomaly is detected, based on the output of the emotion engine. If it determines that the user is experiencing anxiety, it makes adjustments such as softening the notification message.

[0749] Step 5:

[0750] Users check the alerts they receive on their smartphones, pinpoint their child's current location based on the information provided, and view their activity history within the app to understand the situation.

[0751] Step 6:

[0752] Users can contact external organizations such as the police through the app as needed. If the emotion engine determines that the user is in an anxious state, a guide will be added to the notification to help them contact external organizations more smoothly.

[0753] Step 7:

[0754] The device confirms that contact has been made with an external organization, continuously updates its location information in real time, and provides the latest information to the external organization via the server.

[0755] (Example 2)

[0756] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0757] Simply detecting anomalies and issuing alarms to ensure children's safety can cause excessive anxiety among parents. In such situations, cold, emotionless notifications can actually increase anxiety and hinder a swift and appropriate response. Therefore, alarm notifications that take the user's emotional state into consideration are necessary.

[0758] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0759] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, means for modeling the normal range of activity based on the analysis, means for detecting anomalies by comparing with the normal range of activity, means for generating an alarm when an anomaly is detected, means for notifying the user of the alarm, means for communicating with an external organization based on the alarm, means for analyzing the user's emotional state and adjusting the content of the alarm, and means for changing the urgency of the alarm based on emotional information when an anomaly is detected. This enables flexible and effective alarm notifications that respond to the user's emotional state, and is expected to allow for a quick and appropriate response to ensure the safety of children.

[0760] "Location information" refers to data that indicates the geographical location of a specific object or individual.

[0761] "Analysis" is the process of examining collected information and data in detail and extracting patterns and features.

[0762] "Normal range of movement" refers to the area and route that the subject is expected to normally travel, modeled based on past behavioral data.

[0763] "Anomaly detection" is the process of identifying movements that deviate from the normal, expected range of activity or behavioral patterns, and determining that these movements are abnormal.

[0764] An "alert" is a signal or message used to notify a user that attention or caution is necessary when specific conditions or situations occur.

[0765] An "external organization" refers to an organization or group that exists outside the system and with which communication and collaboration are possible.

[0766] "Emotional state" refers to the psychological state or emotional changes an individual is experiencing at a given time.

[0767] "Urgency" is a measure that indicates how quickly a response is required to a particular situation or event.

[0768] This invention is a system designed to ensure the safety of children, utilizing a wearable device, a server, a smartphone application, and an emotion engine. The invention proceeds roughly as follows:

[0769] Use of wearable devices

[0770] The device functions as a wearable device for children. It is equipped with a GPS sensor for acquiring location information and an accelerometer for detecting abnormal movement. The device collects location information and movement patterns in real time and transmits this data to a server. This data is then used to prepare for the detection of deviations from the child's normal range of activity.

[0771] Server Role

[0772] The server analyzes location information based on received data and models the user's normal range of movement. Furthermore, the server detects anomalies by comparing the current location to the normal range of movement. If an anomaly is detected, it generates an alarm and sends it to the user as a notification. In addition, the server is equipped with an emotion engine that analyzes the user's emotional state. Based on the results of this analysis, the urgency and content of the alarm message are adjusted.

[0773] User interaction

[0774] Users can receive alert notifications through a smartphone app and check their child's location and movement history in real time. If necessary, users can contact external organizations, and the system will assist in making such contacts at the optimal time and with the most appropriate content, based on emotional information.

[0775] As a concrete example, a user can input a prompt message to the AI ​​model saying, "When abnormal behavior in a child is detected, please generate a message that will reassure the parent," which will help generate emotionally sensitive messages.

[0776] This system ensures that each function works effectively together, enabling a quick and appropriate response to protect children's safety.

[0777] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0778] Step 1:

[0779] The device acquires location information and movement patterns via a wearable device worn by the child. Specifically, it measures location in real time using a GPS sensor and captures changes in movement using an accelerometer. This data is initially stored locally and then sent to a server. The input is sensor data from the wearable device, and the output is data transmission to the server.

[0780] Step 2:

[0781] The server receives data transmitted from the terminal. It analyzes location and acceleration data and compares it to past patterns to model the normal range of movement. This analysis prepares the server to detect deviations from normal behavior. The input is the data transmitted by the terminal, and the output is the result of the behavior pattern analysis. Based on these results, the server prepares the data for the next step.

[0782] Step 3:

[0783] The server detects anomalies by comparing the normal range of activity with the latest data. When an anomaly is detected, the server flags it as an anomaly and generates an alarm. Specifically, it determines the content and urgency of the alarm based on the type and severity of the abnormal behavior. The input is the result of the behavior pattern analysis, and the output is an instruction to generate an alarm.

[0784] Step 4:

[0785] The server analyzes the user's emotional state using an emotion engine. For example, it identifies anxiety and reassurance by considering alarms and conditioned responses the user has received in the past. This analysis result is used to determine the content and tone of the next alarm message to be generated. The input is interaction data with the user, and the output is the result of the emotional state analysis.

[0786] Step 5:

[0787] The server generates an appropriate alarm message based on the results of anomaly detection and sentiment analysis, and notifies the user. This message incorporates information from the sentiment engine. Upon receiving this notification, the user checks the details via a smartphone app and takes immediate action as needed. The input is the server's analysis result, and the output is the final alarm message sent to the user.

[0788] Step 6:

[0789] Users receive alert notifications through a smartphone app and can check their child's location and movement history. If a user determines that contacting an external agency is necessary, they can use the support prompts built into the app to make contact at the optimal time and with appropriate content, taking emotional information into consideration. The input is an alert message from the server, and the output is the user's action of contacting an external agency.

[0790] (Application Example 2)

[0791] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0792] Ensuring child safety is a critical issue in modern society. However, conventional location-based systems fail to consider the emotional state of parents when detecting anomalies and issuing alarms, potentially causing excessive anxiety. Therefore, there is a need for technology that enables flexible and effective responses that take parents' emotions into account when detecting anomalies and generating alarms based on location information.

[0793] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0794] In this invention, the server includes means for acquiring location information, means for analyzing the acquired location information, and means for adjusting the content of the alarm based on the analyzed emotional state. This makes it possible to issue an appropriate alarm that provides reassurance, taking into account the emotional state of the guardian, when an abnormality in the child's safety is detected.

[0795] "Location information" refers to data that indicates the specific geographical location of an object.

[0796] "Means of analysis" refers to methods or devices for analyzing collected data and extracting useful information from that data.

[0797] "Normal range of activity" is a model of the geographical area that a subject typically travels within on a daily basis.

[0798] "Means for detecting anomalies" refers to methods or devices for identifying movements that deviate from normal behavioral patterns.

[0799] "Means for generating an alarm" refers to a method or device for creating an alert to draw attention when an anomaly is detected.

[0800] "User emotional state" refers to the psychological or emotional situation a user is currently experiencing.

[0801] "Means for adjusting the content of an alarm based on emotional state" refers to a method or apparatus for optimizing an alarm message in accordance with the user's emotional analysis.

[0802] "Means of notification" refers to a method or device for transmitting information or a message to a designated recipient.

[0803] "Means of communication with external organizations" refers to methods or devices that a system uses to exchange information with external organizations or services.

[0804] In the system implementing this invention, a wearable device first acquires the child's location information in real time and transmits it to a server. The wearable device is equipped with a GPS sensor and an accelerometer. The server analyzes the received location information and detects anomalies by comparing it with the child's normal range of activity. At this time, the server uses a machine learning algorithm to model the child's normal range of activity based on past behavioral patterns.

[0805] When an anomaly is detected, the server generates an alarm and notifies the user. Furthermore, it uses an emotion engine to analyze the user's emotional state. The emotion engine utilizes a generative AI model to analyze the user's emotions and provides a means to adjust the content of the alarm based on the results. For example, if the user is feeling anxious, the notification will include a reassuring message.

[0806] Users can check their child's current location and movement history through a smartphone application. Based on this information, they can determine whether it is necessary to contact an external agency. When communicating with an external agency, the server will send a notification based on emotional information at the appropriate time.

[0807] For example, if a child deviates from their usual route to school and is stopped for an extended period at an unexpected location, the server will consider this information suspicious and send a notification to the parent. The AI ​​model then adds considerate wording to the notification for anxious parents, such as, "Your child has been confirmed safe. Please remain calm and check the situation."

[0808] Examples of prompt messages include, "Generate a calm and reassuring notification message based on the user's emotions," and "Create a reassuring notification message for parents if abnormal behavior is detected."

[0809] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0810] Step 1:

[0811] The device uses GPS and accelerometer sensors to acquire the child's location and movement data in real time. This data, including the child's current location and movement patterns, is transmitted from the device to the server. The input is location and velocity data from the sensors, and the output is raw data sent to the server.

[0812] Step 2:

[0813] The server analyzes the received location information and models the normal range of movement by comparing it with past location information. This process uses machine learning algorithms to generate normal behavior patterns and establish criteria for detecting anomalies. The input is location information from the terminal, and the output is a model of the normal range of movement.

[0814] Step 3:

[0815] The server detects an anomaly when location information deviates from the normal range of activity. Upon detection of an anomaly, it generates an alarm and proceeds to the next step. In this step, real-time location data is evaluated based on the generated normal range model to identify the anomaly. The input is real-time location information, and the output is an anomaly detection flag.

[0816] Step 4:

[0817] The server uses a generative AI model to analyze the user's emotions. In this analysis, it detects the user's current emotional state and adjusts the alarm content accordingly. The input is emotional data obtained from the user (e.g., questionnaires for emotion analysis or facial recognition data), and the output is the user's emotional state.

[0818] Step 5:

[0819] If an anomaly is detected, the server adjusts the alarm content based on the user's emotional state and sends a notification to the user's smartphone. This notification includes an emotionally sensitive message, aiming to provide a sense of reassurance. The input is the anomaly detection flag and the adjusted alarm content, and the output is the notification message to the user.

[0820] Step 6:

[0821] Users receive notifications and can use a smartphone application to check their child's location and movement history in real time. They can also consider contacting external organizations if necessary. Input is notification messages from the server, and output is a log of the user's decision-making.

[0822] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0823] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0824] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0825] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0826] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0827] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0828] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0829] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0830] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0831] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0832] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0833] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0834] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0835] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0836] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0837] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0838] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0839] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0840] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0841] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0842] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0843] The following is further disclosed regarding the embodiments described above.

[0844] (Claim 1)

[0845] Means for obtaining location information,

[0846] A means for analyzing acquired location information,

[0847] A means of modeling the normal range of activity based on analysis,

[0848] A means of detecting an anomaly by comparing it with the normal range of activity,

[0849] A means for generating an alarm when an abnormality is detected,

[0850] A means of notifying the user of an alarm,

[0851] A system that includes means for communicating with external organizations based on alarms.

[0852] (Claim 2)

[0853] The system according to claim 1, wherein a machine learning algorithm is used to generate normal behavior patterns during analysis.

[0854] (Claim 3)

[0855] The system according to claim 1, which continuously updates location information in real time when an anomaly is detected.

[0856] "Example 1"

[0857] (Claim 1)

[0858] Means for collecting location data,

[0859] A means of analyzing the collected location data,

[0860] A means for generating a general range of action based on analysis,

[0861] A means of determining abnormalities by comparing them with the normal range of activity,

[0862] A means for generating a notification when an anomaly is detected,

[0863] A means of sending notifications to users,

[0864] A system that includes means for exchanging information with other organizations based on notifications.

[0865] (Claim 2)

[0866] The system according to claim 1, which constructs a general behavioral model using a learning algorithm during analysis.

[0867] (Claim 3)

[0868] The system according to claim 1, which continuously updates location data immediately when an anomaly is detected.

[0869] "Application Example 1"

[0870] (Claim 1)

[0871] Means for obtaining location information,

[0872] A means for analyzing acquired location information,

[0873] A means of modeling the normal range of activity based on analysis,

[0874] A means of detecting an anomaly by comparing it with the normal range of activity,

[0875] A means for generating an alarm when an abnormality is detected,

[0876] A means of notifying users of alarms,

[0877] A means of communicating with external organizations based on an alarm,

[0878] Means of monitoring mobile objects in a home environment,

[0879] Means for controlling the image recording device when an anomaly is detected,

[0880] A means of issuing a warning via audio output in the event of an abnormality,

[0881] A system that includes this.

[0882] (Claim 2)

[0883] The system according to claim 1, which uses a machine learning algorithm to generate normal behavioral patterns during analysis and identifies the location of a moving object within the home.

[0884] (Claim 3)

[0885] The system according to claim 1, which updates location information in real time when an anomaly is detected and records it using an image recording device.

[0886] "Example 2 of combining an emotion engine"

[0887] (Claim 1)

[0888] Means for obtaining location information,

[0889] A means for analyzing acquired location information,

[0890] A means of modeling the normal range of activity based on analysis,

[0891] A means of detecting an anomaly by comparing it with the normal range of activity,

[0892] A means for generating an alarm when an abnormality is detected,

[0893] A means of notifying the user of an alarm,

[0894] A means of communicating with external organizations based on an alarm,

[0895] A means of analyzing the user's emotional state and adjusting the content of the alarm,

[0896] A system that includes means for changing the urgency of an alarm based on emotional information when an anomaly is detected.

[0897] (Claim 2)

[0898] The system according to claim 1, wherein a machine learning algorithm is used to generate normal behavior patterns during analysis.

[0899] (Claim 3)

[0900] The system according to claim 1, which continuously updates location information in real time when an anomaly is detected.

[0901] "Application example 2 when combining with an emotional engine"

[0902] (Claim 1)

[0903] Means for obtaining location information,

[0904] A means for analyzing acquired location information,

[0905] A means of modeling the normal range of activity based on analysis,

[0906] A means of detecting an anomaly by comparing it with the normal range of activity,

[0907] A means for generating an alarm when an abnormality is detected,

[0908] A means of analyzing the user's emotional state,

[0909] A means of adjusting the content of the alarm based on the analyzed emotional state,

[0910] A means of notifying the user of an alarm,

[0911] A system that includes means for communicating with external organizations based on alarms.

[0912] (Claim 2)

[0913] The system according to claim 1, wherein a machine learning algorithm is used to generate normal behavior patterns during analysis.

[0914] (Claim 3)

[0915] The system according to claim 1, which continuously updates location information in real time when an anomaly is detected. [Explanation of Symbols]

[0916] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of obtaining location information, A means for analyzing acquired location information, A means of modeling the normal range of activity based on analysis, A means of detecting an anomaly by comparing it with the normal range of activity, A means for generating an alarm when an abnormality is detected, A means of notifying users of alarms, A means of communicating with external organizations based on an alarm, Means of monitoring mobile objects in a home environment, Means for controlling the image recording device when an anomaly is detected, A means of issuing a warning via audio output in the event of an abnormality, A system that includes this.

2. The system according to claim 1, which uses a machine learning algorithm to generate normal behavior patterns during analysis and identifies the location of a moving object within the home.

3. The system according to claim 1, which updates location information in real time when an anomaly is detected and records it using an image recording device.

Images